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Introduction 


The third edition of the Global inventory of regional and 
national qualifications frameworks is published at a time 
when political attention to national and regional qualifi- 
cations frameworks is increasing. UNESCO has adopted 
the Education 2030 agenda as a framework for action 
that supports implementation of the sustainable devel- 
opment goal 4 (SDG 4), focusing on promoting inclusive 
and equitable education as well as lifelong learning oppor 
tunities for all. Learning-outcomes-based qualifications 
frameworks can contribute directly to this goal. In June 
2016, the European Commission has adopted the New 
Skills agenda for Europe with actions aiming to improve 
the quality and relevance of skills for citizens and the la- 
bour market. The Skills agenda underlines the need to 
continue developing the European qualifications frame- 
work (EQF), noting the progress made after 2008. The 
revised EOF (adopted 22 May 2017) is expected to play a 
key role in taking forward EU skills and lifelong learning 
strategies, directly aiding transparency and portability of 
qualifications in Europe. New regional frameworks are 
now becoming operational. The Association of Southeast 
Asian Nations (ASEAN) qualifications reference frame- 
work (AORF) is the first regional qualifications framework 
(after the EQF) to attain operational status (2017). Initi- 
atives in Africa, the Caribbean and the Pacific area also 
illustrate this tendency. 


More than 150 countries worldwide are now developing 
and implementing qualifications frameworks. The num- 
ber of frameworks has not grown much from the last 
edition of the global inventory in 2015, but, although 
scattered, there has been a certain deepening of activi- 
ties. Many countries around the globe see learning-out- 
comes-based national qualifications frameworks (NOFs) 
as instruments for supporting reforms, as a way to im- 
prove transparency and relevance of qualifications, and 
as a way to open opportunities to validation of wider 
access to learning opportunities and pathways. Being 
developed in different contexts and for varying purpos- 
es, qualifications frameworks are not a panacea for all 
the problems that countries face. Country case studies 
outline several key factors that shape successful devel- 
opment and implementation. While the technical, con- 
ceptual and legal bases of NOFs are important, country 
cases demonstrate that commitment and ownership are 
crucial preconditions for success; stakeholders need to 
buy into the frameworks and use them. 
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Scope of the inventory 


The global inventory 2017 further strengthens the knowl- 
edge base on developments in national and regional qual- 
ifications frameworks, as implemented by countries and 
regions, by end of 2016. It acts as an observatory of pro- 
gress in establishing NOFs, as well as the challenges and 
success factors in implementation. The thematic chapters 
(Volume |) discuss key trends and policy issues emerging 
from qualifications frameworks and learning outcomes re- 
forms and developments. Volume || consists of NOF case 
studies of 100 countries from all continents and seven re- 
gional qualifications frameworks (ROF). 


The case studies look at the educational, social, eco- 
nomic and political context In which an NOF is embed- 
ded, main policy objectives, implementation of learning 
outcomes, stakeholder involvement and_ institutional 
arrangements. They also show how NOFs open for and 
link to validation arrangements, support recognition and 
learning pathways. They conclude with important les- 
sons and future plans. 


The global inventory draws on inventories from four 
agencies: the European Centre for the Development 
of Vocational Training (Cedefop), the European Training 
Foundation (ETF), the United Nations Educational, Scien- 
tific and Cultural Organisation (UNESCO) and the UNE- 
SCO Institute for Lifelong Learning (UIL), working and 
communicating with country officials and experts. It also 
draws on extensive international research in qualifica- 
tions reforms, comparability of qualifications, and use of 
learning outcomes. 


Thematic focus 


The 2017 inventory shows that countries around the 
globe are reforming their national qualification systems 
to improve the readability, quality and relevance of qual- 
ifications. Learning-outcomes-based national qualifica- 
tions frameworks (NOFs) are playing an increasingly 
important role in these reforms. The thematic chapters 
are structured around the three strands related to the 
purposes and impact of frameworks, the role of frame- 
works as catalysts for lifelong learning, and the role of 
frameworks in promoting international and global com- 
munication and cooperation. 
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The purposes and impact 

of qualifications frameworks 

Qualifications frameworks function within dynamic and 
complex national education and training systems and la- 
bour markets. They are embedded in, and shaped by, 
broader developments and objectives and their success 
and impact depends on their ability to engage with these. 
This perspective is pursued by Borhene Chakroun (Chap- 
ter 1) exploring the role of qualifications frameworks in ad- 
dressing the Sustainable development agenda adopted by 
the United Nations in 2015. Chakroun argues that NOFs 
cannot be defined according to a narrow set of economic 
objectives but need to be understood as instruments facing 
a multifaceted reality and contributing to objectives of eco- 
nomic growth, social equity and sustainable development. 
The author argues that the learning outcomes perspective 
is of particular importance in addressing this multifaceted 
reality. Learning outcomes introduce a common language 
allowing for communication, cooperation and coordination 
between diverse stakeholders across institutional, sectoral 
and national borders. This is a precondition for improving 
the quality and relevance of qualifications and qualifica- 
tions systems and lifelong learning opportunities for all (as 
stated in sustainable development goal 4). 


Referring to the experiences of the 29 ETF partner coun- 
tries in the Western Balkans and Turkey, Eastern Europe 
and Central Asia and the Middle East and North Africa, 
Michael Graham and Arjen Deij (Chapter 2) point to con- 
ditions to be met for an NOF to become operational and 
take on the roles and functions targeted by a country. For 
an NOF to become an integrated and operational part of 
the national qualifications system a legal basis has to be 
established, stakeholder involvement must be ensured, 
and institutional structures and quality assurance mecha- 
nisms must be in place. Politically and institutionally isolat- 
ed frameworks will not be able to engage with stakehold- 
ers and established structures and will fail to achieve their 
potential or add value. 


Building on European experiences and taking into account 
international experiences during the past decade, Slava 
Pevec Grm and Jens Bjernavold (Chapter 3) address the 
challenge of measuring the impact of NOFs. The chapter 
identifies several areas where NOFs in Europe are start- 
ing to make a difference but also areas where impact is 
less visible. They argue that institutional robustness, use of 
the frameworks by powerful stakeholders and visibility to 
end-users are critical preconditions for a framework to add 
value. A key challenge is related to setting an assessment 


baseline reflecting the multifaceted character of the NOFs 
and the contexts and reality in which they operate. The 
authors argue that the future of NOFs depends on their 
proven ability to add value, placing impact assessment at 
the core of future NOF developments. 


Qualifications frameworks as catalysts 

for lifelong learning 

A key purpose of national qualifications frameworks is 
to support learning across different areas and through- 
out life. Madhu Singh (Chapter 4) discusses what this 
entails in practice elaborating what is required, in terms 
of adaptation and flexibility, by a ‘quality lifelong learn- 
ing system’. Singh argues that such a system must be 
able to support diverse learning forms (formal, non-for 
mal informal) and address the needs of a wide variety of 
users. The chapter lists and discusses several elements 
necessary for making such a diverse system operational. 
It also details how national qualifications frameworks, the 
learning outcomes principle, qualifications standards and 
mechanisms for recognition (validation) of non-formal and 
informal learning can interact to open up to more flexible, 
lifelong-learning-oriented-pathways for learning. 


A lifelong learning strategy must make visible and at- 
tribute value to the learning taking place outside formal 
education and training systems. National qualifications 
frameworks play a role in aiding these efforts. Ernesto 
Villalba and Jens Bjarnavold (Chapter 5) show how val- 
idation of non-formal and informal learning is gradually 
becoming an integrated part of national lifelong learning 
arrangements in Europe. Although most European coun- 
tries now officially have clear policy intentions in this field, 
this is not always translated into practical arrangements 
giving citizens access to validation. The chapter explores 
the main conditions that must be fulfilled for validation to 
be ‘mainstreamed’, notably focusing on stable institution- 
al arrangements, sufficient human and financial resourc- 
es and trusted validation methodologies. 


Quality assurance can be seen as an important lifelong 
learning building block and is intrinsically linked to the 
development and implementation of qualifications frame- 
works. This topic, particularly how quality assurance can 
support the development of learning-outcomes-based 
qualifications, is covered by Borhene Chakroun and George 
Kostakis (Chapter 6). The chapter presents recent trends in 
this area and discusses some of the main challenges when 
developing effective and trustworthy quality assurance of 
qualifications. The authors note that a broad consensus ex 


10 


ists around the idea that quality assurance is crucial for 
ensuring trust in qualifications. Based on experiences from 
technical and vocational education and training, the chap- 
ter addresses the implications of a shift from an input- to 
an outcome-oriented quality assurance perspective. 


Qualifications frameworks promoting international and global 
communication and cooperation 

It is commonly argued that qualifications frameworks 
play a key role in increasing the cross-border transpar 
ency of qualifications. This directly supports student and 
worker mobility, learning and careers. Jens Bjornavold 
and Borhene Chakroun (Chapter 7) discuss the challeng- 
es and opportunities involved in comparing technical and 
vocational education and training (TVET) qualifications. 
The chapter summarises the outcomes of joint research 
carried out by Cedefop, the ETF and UNESCO (in 2015-16) 
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comparing the content and profile of four vocational qual- 
ifications in 26 countries worldwide, using the Europe- 
an classification of skills, competences, occupations and 
qualifications (ESCO) as a reference point. The authors 
note the usefulness of levelling through qualifications 
frameworks, but underline that comparability of qualifi- 
cations requires in-depth studies like the one tested by 
Cedefop, the ETF and UNESCO. 


The thematic chapters are concluded with a presentation 
(Chapter 8) of continuing work, initiated by UNESCO, on 
world reference levels to Support cross-border mobility of 
learners and workers. Written by Borhene Chakroun and 
Katerina Ananiadou, this discussion is informed by a se- 
ries of studies that are expected to clarify the character 
and function of these reference levels and define their ra- 
tionale and ecosystem. 
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CHAPTER 1. 
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QUALIFICATIONS FRAMEWORKS IN A SUSTAINABLE 
DEVELOPMENT CONTEXT: REFLECTIONS AND PERSPECTIVES 


Borhene Chakroun, UNESCO 


Summary 


This chapter explores the relationship between the Sus- 
tainable development agenda and qualifications frame- 
works. It aims at demonstrating the mutually reinforcing 
nature of the Education 2030 agenda and qualifications 
frameworks goals. The chapter starts with a presentation 
of the Sustainable development agenda including the 
Education 2030 agenda and the place of qualifications in 
this context. It shows that the agenda proposes new per 
spectives for defining and analysing national qualifications 
frameworks (NOFs) development objectives. It also con- 
siders that sustainable development goals (SDGs) offer a 
fresh look at how we measure and assess the impact of 
NOFs both for individual learners and for societies. Finally, 
we demonstrate that the use of learning outcomes stands 
out as one of the most important convergence features of 
both the Education 2030 agenda and NOFs. 


Keywords: qualifications; sustainable development; learn- 
ing outcomes 


1.1. Introduction 


The international community has set an ambitious 2030 
agenda for sustainable development. This agenda is a plan 
of action for people, the planet and prosperity. All countries 
from all income levels, acting in partnership, are expected to 
implement this plan. The 17 sustainable development goals: 
(Table 1-1) and 169 targets adopted demonstrate the scale 
and ambition of this new universal agenda. They focus on 
five key elements: people, planet, peace, prosperity, and 
partnership. They seek to build on the Millennium develop- 
ment goals and complete what these did not achieve. They 
are integrated and indivisible and balance the three dimen- 
sions of sustainable development: the economic, social and 
environmental (United Nations, 2015). 


In this context, sustainable development goal (SDG) 4 
seeks to ‘ensure inclusive and equitable quality educa- 
tion and promote lifelong learning for all’. Looking across 
the 17 goals and 169 targets there are numerous areas 


where, in addition to SDG 4, education and training have 
a role to play if the SDGs are to be achieved. They include 
poverty reduction (goal 1), agricultural productivity (goal 
2), clean energy (goal 7), sustained, inclusive and sustain- 
able economic growth with full and productive employ- 
ment and decent work for all (goal 8) and the promotion 
of inclusive and sustainable industrialisation and fostering 
innovation (goal 9). 


1.2. Qualifications in the context of Education 2030 


A framework for action supports the implementation of 
SDG 4. It includes targets, indicative strategies and a se- 
ries of indicators helping policy-makers and national stake- 
holders to relate to the challenges at hand (Table 1-2). 


Five of the 10 targets are concerned with improving the 
quality of education for individual children, young people 
and adults, and to give them better and more relevant 
knowledge and skills. This emphasis on learning outcomes 
is a timely step forward from the Education for all (EFA) 
and the Millennium development goals (MDG), which fo- 
cused mainly on ensuring access to primary education. 


Qualifications are also at the centre of the targets relat- 
ed to vocational and tertiary education. The emphasis is 
on increasing the relevance and flexibility of education and 
training programmes, enhancing lifelong learning, improv- 
ing the transparency of qualification systems, creating 
possibilities for credit accumulation and transfer, or devel- 
oping quality assurance systems. It is also significant that 
not only national developments are considered in SDG 4; 
cross-border recognition of qualifications and worker and 
learner mobility are also highlighted (see Table 1-3 for re- 
lated indicative strategies). Improving the transparency of 
qualifications and lifelong learning are two fundamental el 
ements of efforts to bring education and training into line 
with SDG 4 and more broadly the SDGs. 


While the indicative strategies emphasise that qualifica- 
tions need to fit in a country’s own context, they are also 
suggesting that national qualifications frameworks cannot 
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Table 1-1. Sustainable development goals 


Ens SUSTAINABLE DEVELOPMENT GOAL 


End poverty in all its forms everywhere 


— THEMATIC CHAPTERS 


2 End hunger, achieve food security and improved nutrition and promote sustainable agriculture 

3 Ensure healthy lives and promote well-being for all at all ages 

4 Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all 

5 Achieve gender equality and empower all women and girls 

6 Ensure availability and sustainable management of water and sanitation for all 

7 Ensure access to affordable, reliable, sustainable and modern energy for all 

8 Promote sustained, inclusive and sustainable economic growth, full and productive employment and 
decent work for all 

9 Build resilient infrastructure, promote inclusive and sustainable industrialisation and foster innovation 

10 Reduce inequality within and among countries 

11 Make cities and human settlements inclusive, safe, resilient and sustainable 

12 Ensure sustainable consumption and production patterns 

13 Take urgent action to combat climate change and its impacts (*) 

14 Conserve and sustainably use the oceans, seas and marine resources for sustainable development 
15 Protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, 
combat desertification, and halt and reverse land degradation and halt biodiversity loss 
16 Promote peaceful and inclusive societies for sustainable development, provide access to justice for all 

and build effective, accountable and inclusive institutions at all levels 
17 Strengthen the means of implementation and revitalise the global partnership for sustainable 


development 


(*) Acknowledging that the United Nations framework convention on climate change is the primary international, intergovernmental forum for 


negotiating the global response to climate change. 


Source: United Nations, 2015. 


be developed in isolation from those in other countries. 
The increased transparency — and underlying trust — should 
also serve the international mobility of students and work- 
ers and the increasingly linked labour markets. 


The focus on NOFs is important because, according to the 
present global inventory, around 150 countries are now 
involved in some way in reforming their qualifications sys- 
tems. Most of these countries see an NOF as a way of im- 
proving the quality of their education and training provision 
and raising the skill levels of their workforce (Allais, 2010) 
which they expect will lead to work becoming more pro- 


ductive and decent and their economies becoming more 
competitive and sustainable. 


In this context, national qualifications systems are in- 
creasingly associated with the Sustainable develop- 
ment agenda, particularly SDG 4. These could lead to 
a more generalised form of NOFs, including common 
characteristics such as the use of learning outcomes, 
quality assurance, recognition and validation policies, 
but this does not necessarily imply conformity of NOFs 
across countries. 
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Table 1-2. SDG 4 targets 


TARGET NUMBER _ | DESCRIPTION OF THE TARGET 


By 2030, ensure that all girls and boys have access to quality early childhood development, 


including technical and vocational skills, for employment, decent work and entrepreneurship 


By 2030, ensure that all youth and a substantial proportion of adults, both men and women, 


development and sustainable lifestyles, human rights, gender equality, promotion of a culture 


countries, in particular least developed countries, small island developing States and African 
countries, for enrolment in higher education, including vocational training and information and 
communications technology, technical, engineering and scientific programmes, in developed 


TARGET 4.1 By 2030, ensure that all girls and boys complete free, equitable and quality primary and 
secondary education leading to relevant and effective learning outcomes 

TARGET 4.2 
care and pre-primary education so that they are ready for primary education 

TARGET 4.3 By 2030, ensure equal access for all women and men to affordable and quality technical, 
vocational and tertiary education, including university 

TARGET 4.4 By 2030, substantially increase the number of youth and adults who have relevant skills, 

TARGET 4.5 By 2030, eliminate gender disparities in education and ensure equal access to all levels 
of education and vocational training for the vulnerable, including persons with disabilities, 
indigenous peoples and children in vulnerable situations 

TARGET 4.6 
achieve literacy and numeracy 

TARGET 4.7 By 2030, ensure that all learners acquire knowledge and skills needed to promote 
sustainable development, including, among others, through education for sustainable 
of peace and non-violence, global citizenship and appreciation of cultural diversity and of 
culture's contribution to sustainable development 

TARGET 4.A Build and upgrade education facilities that are child, disability and gender sensitive and 
provide safe, non-violent, inclusive and effective learning environments for all 

TARGET 4.B By 2020, substantially expand globally the number of scholarships available to developing 
countries and other developing countries 

TARGET 4.C 


By 2030, substantially increase the supply of qualified teachers, including through 


international cooperation for teacher training in developing countries, especially least 
developed countries and small-island developing States 


Source: UNESCO, 2015a. 


The presentation, so far, of the place of qualifications 
in SDG 4 can be used both to open up the scope of 
possibilities for the NOFs (‘what is an NOF for?’), and 
offer an exploratory analysis of the catalyst role of 
learning outcomes for systemic reforms (‘can learn- 
ing outcomes act as a common language for lifelong 
learning?’). The next two sections explore these issues 
and demonstrate that SDG 4 and NOF efforts can be 
mutually reinforcing. NOFs can support progress in 
achieving SDG 4. In turn, SDG 4 can reinforce the role 
of NOFs as tool for stakeholder cooperation and sys- 
temic lifelong learning reform. 


1.3. What is an NOF for? A shift from neoliberal to 
sustainable development paradigm 


From a study of policy documents, Allais (2010) noted 
that NOFs are seen as a solution to many of the prob- 
lems within education and training systems. The author 
considers that NOFs can contribute to achieving two 
significant development goals and rationales: social eq- 
uity (by encouraging participation in raising education- 
al achievement and developing more diverse learning 
pathways); and economic development (by linking qual- 
ifications more strongly to the labour market and com- 
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Table 1-3. Indicative strategies focusing on qualifications in SDG 4 


TARGET 4.3: BY 2030, ENSURE EQUAL ACCESS FOR ALL WOMEN AND MENTO AFFORDABLE 


AND QUALITY TECHNICAL, VOCATIONAL AND TERTIARY EDUCATION, INCLUDING UNIVERSITY 


Develop cross-sector policies for and between vocational skills development, TVET and tertiary education and 
strengthen links between science and policy development to keep pace with changing contexts and remain relevant; 
develop effective partnerships, in particular between the public and private sectors, and include employers and 
unions in their implementation. 


Ensure quality assurance, comparability and recognition of tertiary education qualifications and facilitate credit 
transfers between recognised tertiary education institutions. 


Develop policies and programmes for the provision of quality distance learning in tertiary education, with appropriate 
financing and use of technology, including the Internet, massive open online courses and other modalities that meet 
accepted quality standards to improve access. 


Develop policies and programmes that reinforce the research function in tertiary and university education through 
the early uptake of the STEM fields, particularly by girls and women. Strengthen international cooperation in 
developing cross-border tertiary and university education and research programmes, including within the framework 
of global and regional conventions on the recognition of higher education qualifications, to support increased access, 
better quality assurance and capacity development. 


Promote TVET, tertiary education and university as well as adult learning, education and training opportunities for 
young people and adults of all ages and sociocultural background so as to enable them to continue to improve and 
adapt their skills, with particular attention to gender equality including the elimination of genderbased barriers, and 
to vulnerable groups such as those with disabilities. 


Tertiary institutions, including universities should support and foster the development of policies for and provision of 
equitable quality lifelong learning opportunities. 


TARGET 4.4: BY 2030, SUBSTANTIALLY INCREASE THE NUMBER OF YOUTH AND ADULTS WHO HAVE 
RELEVANT SKILLS, INCLUDING TECHNICAL AND VOCATIONAL SKILLS, FOR EMPLOYMENT, DECENT 


WORK AND ENTREPRENEURSHIP 


Gather and use evidence about changing skills demand to guide skills development, reduce disparity and 
respond to changing labour market and societal needs and contexts, as well as to the needs of the ‘informal 
economy’ and rural development. 


Engage social partners in designing and delivering education and training programmes that are evidence-based 
and holistic. Ensure that TVET curricula and training programmes are of high quality and include both work-related 
skills and non-cognitive/transferable skills, including entrepreneurial, basic and ICT skills, and that TVET institutions’ 
leaders and teaching staff, including trainers and companies, are qualified/certified. 


Promote the development of different forms of work-based and classroom-based training and learning where 
appropriate. Ensure transparent and efficient TVET quality assurance systems and develop qualifications frameworks. 


Promote collaboration on enhancing transparency and cross-border recognition of TVET qualifications to raise the 
quality of TVET programmes and enable workers’ and learners’ mobility, and to ensure that TVET programmes keep 
pace with the changing labour market demands. 


Promote flexible learning pathways in both formal and non-formal settings; enable learners to accumulate and 
transfer credits for levels of achievement; recognise, validate and accredit prior learning; and establish appropriate 
bridging programmes and career guidance and counselling services. 


Source: UNESCO, 2015a. 
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petitiveness) (idem, p. 17). However, as noted by Coles 
and colleagues (Coles et al., 2014, p. 21), economic driv- 
ers clearly dominated the pressures for creating an NOF 
and using it to improve the volume, focus and quality of 
training. According to the authors (idem), linked to this 
was the need for enterprises to be competitive globally 
and for countries to be attractive as places to invest in 
business infrastructure. 


The economic emphasis of NOFs was forcefully criticised 
and linked to neoliberal public sector reform. Philipps 
(2003), for instance, considers that qualification reform in 
New Zealand had an economic focus from the start, re- 
sulting directly from the public sector reforms that empha- 
sised a market-based economy. Young (2003, p. 232 cited 
in Allais 2010, p. 24) strongly suggests that qualifications 
frameworks represent an ‘almost paradigm case of gov- 
ernment intervention in a neoliberal economy’, as they are 
attempts both to gain greater central control and to give 
greater choice to individuals. The response to economic 
demands should not be at the expense of the other cate- 
gories of demands as this can result in an unbalanced and 
distorted education and training system, particularly TVET 
(see UNESCO, 2015a). It is necessary to ensure that the 
analysis is not overly dominated by one demand or anoth- 
er; if it is, this should be a strategic choice rather than an 
omission (idem). 


National qualifications frameworks are increasingly marked 
by the interest of governments in integrating education 
and training, developing lifelong-learning-oriented frame- 
works that incorporate qualifications from different educa- 
tion and training sectors (general, vocational and academ- 
ic) and that are often open to the learning taking place in 
different settings (Cedefop, 2016; UNESCO, 2015a). They 
go beyond the economic dimension and aim at redefinition 
of the way qualifications are valued and eventually put into 
use in societies. NOFs, as Bjornavold and Coles note in 
European Commission 2009, can be expected to act as 
‘instruments with a vision’. 


Renewed attention to the purpose and relevance of educa- 
tion for human development and economic, social and en- 
vironmental sustainability is a defining feature of the SDG 
4 -Education 2030 agenda. This is embedded in its holistic 
and humanistic vision, which contributes to a new model 
of human development (UNESCO, 2015a). In this context, 
the vision of NOFs should be related to the wider Sus- 
tainable development agenda. This means that economic 
growth and, more broadly, prosperity must be conceived 
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in ways that leave no one behind. Closer integration of ed- 
ucation, economic and employment policies are essential 
for that change to happen (UNESCO, 2016). 


SDGs provide an opportunity for better measuring the im- 
pact of NOFs building on existing efforts and taking into 
consideration the sustainable development three key pil- 
lars: economic growth; social equity; and environmental 
sustainability. The three pillars offer a fresh look at how we 
can measure and assess the impact of NOFs both for indi- 
vidual learners and for societies. 


UNESCO (2015b) proposed a new, integrated, analytical 
approach to analysing education and training policies that 
combines economic growth, social equity and sustaina- 
bility concerns in a balanced and strategic manner. Each 
of these three development areas is conceived as an 
‘analytical lens’ through which to view a country’s edu- 
cation and training systems, including national qualifica- 
tions frameworks. In summary these ‘lenses’ (UNESCO, 
2015b, p. 149) are: 


(a) economic growth lens, incorporating productivity and 
growth, employability, employment creation and new 
modes of work; 

(b) social equity lens, incorporating redistribution of both 
material and intangible wealth and inclusion; 

(c) sustainability lens, incorporating greening economies, 
intergenerational rights and global citizenship. 


The three lenses approach provides a strong analytical 
framework for considering the purpose of education and 
training, particularly TVET, including qualifications system 
reform efforts. The relative ‘success’ of NOF policy and sys- 
tems reform can be assessed by the extent to which such 
reform achieves positive outcomes related to these three 
lenses. The benefit of the analytical approach proposed 
here is that it provides a holistic way of assessing how a 
particular policy tool such as NOFs is related to wider de- 
velopment demands. The most recent attempt to evaluate 
NOF impact in South Africa has paid great attention to the 
broader development context of the country (SAQA, 2017). 
The evaluation has been framed in the country's National 
development plan 2030 (NDP) and the Human resource 
development strategy for South Africa (HRD-SA), 2013-30. 


However, the multisectoral character of NOFs and their ca- 
pacity to respond simultaneously to the external demands 
of economic growth, social equity and sustainability, is often 
hampered by the lack of complementarity and alignment 


16 


with other policies (Mukora 2006; Raffe, 2009; Chakroun 
and Jimeno, 2010). They tend to lack what Raffe (2009) re- 
fers to as ‘policy breadth’, and are insufficiently linked with 
other education policy measures that need to be addressed 
in the process of reform, such as funding, teacher training, 
and the autonomy of education institutions. 


While a fragmented approach to policy-making and a lack 
of coordination across ministries has been identified as a 
contributory factor to the ‘failure’ of NOFs to achieve their 
potential, a more fundamental problem is the absence of 
a suitable, shared and intersectoral approach which could 
help to connect the analysis of NOFs with intended devel- 
opment outcomes. 


UNESCO's new recommendation on TVET (UNESCO, 
2015c) suggests that countries should develop TVET polli- 
cies that are consistent with a broad range of policy fields, 
including employment, industry development, and agricul- 
ture, as well as the overall strategic objectives of govern- 
ments, particularly their economic, social and environmen- 
tal objectives. This applies as well to NOFs as a policy tool 
for reforming education and training systems. 


These policy objectives (economic, social and environ- 
mental) each have a role to play in shaping national skills 
systems. However, as noted by UNESCO (2015b, p. 172), 
each of the three areas has its own group of stakeholders. 
These groups have developed their own outlooks, theo- 
ries, methodologies, and specialised languages. They also 
have distinctive priorities, needs, interests and positions, 
which may or may not coincide, and which can bring pol- 
itics and power games into education and training policy 
processes (ibid). 


Hence, the implementation and impact of a framework will 
depend on Its alignment with national policy, institutional 
priorities and other contextual pressures, plus cooperation 
among national stakeholders. This is what the Sustainable 
development agenda and Education 2030 are bringing. 
SDGs are integrated and indivisible and balance the three 
dimensions of sustainable development. They emphasise 
the importance of partnership, coordination and coopera- 
tion and interlinkages of public policies and what OECD 
calls ‘whole government approach’ (OECD, 2011). 


The integrated nature of the sustainable development 
goals are of crucial importance for NOF ‘policy breadth’ 
and coordination. Those governments which are moving 
ahead with the implementation of the Sustainable devel- 
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opment agenda and which are able to establish coordina- 
tion and coherence between policy domains so that they 
inform and complement each other should be inherently 
better placed to drive NOFs implementation and achieve 
more impact on individuals, economies and societies than 
where these domains conflict and operate in isolation (UN- 
ESCO-ILO, forthcoming). 


Placing discussion of NOF objectives in the context of 
SDGs provides an opportunity for better balancing the 
economic, social equity and environmental sustainability 
rationales of NOFs. This, in turn, will have an influence on 
the way NOF impact is assessed. The attention SDGs give 
to cooperation and partnership, monitoring and evaluation, 
including definition of targets and indicators and adoption 
of reliable systems that collect and provide access to time- 
ly, comprehensive and forward looking data, are promising 
for NOF impact analysis. 


The global effort to monitor learning achievements is an- 
other aspect that can reinforce the application of a learning 
outcomes approach promoted by NOFs and the way they 
are considered in education and training. 


1.4. Can learning outcomes act as ‘common 
language’ for lifelong learning? 


Learning outcomes are increasingly used in describing 
curricula, qualifications, assessment processes and NOF 
levels (Cedefop, 2016). Internationally the use of different 
forms of level descriptors shows diverse and contempo- 
rary learning-outcomes-based developments from across 
education, training and work (Keevy and Chakroun, 2015; 
Coles, 2017). 


Several factors put learning outcomes high on the policy 
agenda: the shift from focus on access to outcomes in the 
international education agenda and the attention to wheth- 
er students are learning and acquiring relevant skills; adop- 
tion of lifelong learning strategies; development of national 
qualifications frameworks; and the introduction of mech- 
anisms for recognising and accrediting non-formal and in- 
formal learning. 


The term ‘learning outcomes’ is embedded in the vocab- 
ulary of the Education 2030 agenda. The definitions of the 
concept, however, vary across contexts and education 
subsectors, although common elements can be identified: 
the totality of information, knowledge, understanding, at- 
titudes, values, skills, competencies or behaviours an indi- 
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vidual is expected to master upon successful completion 
of an education programme (ISCED, 2011); and statements 
of what a learner knows, understands and is able to do on 
completing learning (Cedefop, 2009). In both cases, learn- 
ing outcomes include three major domains: knowledge 
(learning to know), skills (learning to do) and competences 
(learning to be and to live together). 


Compared with Education for all (EFA), Education 2030 
marks a shift towards learning outcomes. Yamada (2016, 
p. 30) notes three significant shifts. First, the quality of 
education, which used to be monitored by access as well 
as the amount of inputs to education and training systems 
such as facilities, textbooks, and teachers, is now meas- 
ured by the amount and the type of knowledge, skills and 
abilities of learners. Second, the outcomes of learning are 
considered to be the improved capacities of learners to 
adapt knowledge to daily contexts and to solve problems, 
not simply the abilities to recite the contents of curriculum. 
Third, the domains of learning outcomes are not restricted 
by the framework of curricular subjects, but are cross-cut- 
ting and broad in nature. 


NOFs reforms have paved the way for attention to learn- 
ing outcomes and for learning in different settings: formal, 
non-formal and informal. NOFs also reinforce the valida- 
tion of learning gained outside formal education provision. 
Learning-outcomes-based qualifications challenge as- 
sumptions that the location of learning is only in formal 
education and training providers. It reinforces the idea that 
learning occurs in work and social life. This strengthens the 
link between education provision and the labour market 
and offers greater lifelong learning opportunities for those 
who have faced barriers in accessing and completing for 
mal education and training programmes. 


As illustrated by this global inventory, there are indica- 
tions that the learning-outcomes-based national qualifi- 
cations frameworks are playing an increasingly important 
role in education and training system reform (Cedefop 
2016; Keevy and Chakroun, 2015; UNESCO, 2015b). An 
increasing number of countries report that they use the 
frameworks to check the consistency of qualifications, 
seeking to clarify levels, avoid overlaps and aid linkages 
and progression. 


However, outcomes-based qualifications frameworks 
and qualifications reforms based on learning outcomes 
might interact with the systems differently in different 
contexts, for example in developed and developing 
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countries (Young, 2011). The former normally have strong 
education institutions, well trained teachers and estab- 
lished partnerships between education and the labour 
market while the latter are, in many cases, struggling 
with reforming curricula, establishing new institutions 
and changing governance to include new stakeholders 
such as the social partners. It is also wrong to assume 
that all aspects and facets of learning can be captured 
through learning outcomes. Cedefop's handbook on 
learning outcomes (Cedefop, 2017) notes that learning 
outcomes cannot stand alone; their potential can only 
be released when interacting with teaching, learning 
and assessment. 


1.4.1. Improving learning assessment 

SDG 4 and NOF efforts can be mutually reinforcing in im- 
proving learning assessment. Both point to attention to 
develop appropriate standards, to map and compare the 
actual achievements of learners demonstrated through as- 
sessments. NOFs focus further on the use of skills at work, 
which is also covered in target 4.4 of SDG 4 (Table 1-2). 


In the context of SDG 4, the current spotlight on learning 
assessments (national, regional and international) opens up 
an important window of opportunity to discuss the deeper 
transformations needed in assessment systems worldwide. 


Qualifications frameworks, with their emphasis on 
learning outcomes, have the potential to advance the 
Agenda for sustainable development and may provide 
the reference point and baseline required for measuring 
progress in accompanying this shift. This includes re- 
view of the assessment criteria and methods, responsi- 
bility for defining assessment criteria, the stakeholders 
involved, including internal and external dimensions of 
assessment, centralisation/decentralisation of assess- 
ment, and the involvement of private sector and civil 
society representatives. 


Concurrent with the rise of learning outcomes is the in- 
creasing use of cross-national and national learning as- 
sessments such as PISA, PIAAC, TIMSS, PIRLS, SAC- 
MEQ and PASEC. While these surveys focus mainly on 
foundation skills, mathematics and sciences, they have 
also attempted to assess other transferable skills, such 
as problem solving. Attention is also being paid to digital 
skills. This drive to benchmark global progress in improving 
education outcomes, and to increase the evidence base 
for education policy-making reinforces NOF attention to 
learning outcomes. 
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Global dialogue may help emphasise the need for more 
inclusive, and lifelong-learning-oriented, assessment poli- 
cies in education and training, enabling a greater diversity 
of learners to have their skills recognised and validated, 
including through validation of prior learning processes. 


1.4.2. Expanding the domains of learning-outcomes 

There is growing international evidence that a new set 
of transversal skills, more suited to the needs of knowl- 
edge-rich and digital economies, are emerging (OECD, 
2016). Typically, these include creativity, critical thinking, 
collaboration, and communication. Added to them is the 
fact that market-driven economies increasingly value en- 
trepreneurship. Yet, SDGs highlight that skills formation 
should not be driven exclusively by economic needs but 
by the constant quest of individuals and societies to live 
better ina much more complex world where sustainability, 
equity, and global citizenship are key. This has implications 
for descriptors for defining learning outcomes in NOFs and 
may shift the emphasis from a behaviouristic approach to 
more socio-constructivist approach of defining levels de- 
scriptors and learning outcomes (Keevy and Chakroun, 
2015; Cedefop, 2016). 


The learning domains under SDG 4 encompass cognitive, 
non-cognitive and behavioural skills. Target 4 of SDG 4 
highlights skills for the workplace, target 6 the literacy and 
numeracy skills, and target 7 the values and attitudes to 
live in a more sustainable world. Discussions on learning 
domains in the context of SDG 4 may contribute to the 
development of a broader spectrum of technical and trans- 
ferable skills recognised as highly relevant in the world of 
work and for life, contrasting with a narrow focus on job-re- 
lated skills (see Keevy and Chakroun, 2015 and Chapter 7 
of this volume). 


1.4.3. Learning outcomes as common language for lifelong 
learning and cooperation 

Strengthening the linkages between education subsectors 
in a lifelong learning perspective is an explicit goal of Edu- 
cation 2030. The aim is to promote quality lifelong learning 
opportunities for all, in all settings and at all levels of edu- 
cation (UNESCO, 2016). Education 2030 goes beyond past 
attempts to ensure access to basic education, as set out 
in the goals of Education for all and the education-related 
Millennium development goal 2 (MDG 2) of 2000-15. It ex 
panded scope and reaches from early childhood learning 
to youth and adult education and training; it aims to ensure 
quality learning outcomes for all, throughout life. Educa- 
tion 2030 considers that: ‘lifelong learning for all, in all set- 
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tings and at all levels of education, should be embedded 
in education systems through institutional strategies and 
policies, adequately resourced programmes, and robust 
partnerships at the local, regional, national and internation- 
al levels’ (p. 33). 


The development of comprehensive national qualifica- 
tions frameworks that include all education and training 
subsectors and qualifications has the potential to create a 
favourable policy environment in most countries for coop- 
eration, communication and mutual learning between ed- 
ucation subsectors (Cedefop, 2016). The attention of SDG 
4 to learning outcomes and the importance given to them 
in qualifications reforms points towards a ‘common lan- 
guage’, allowing for dialogue between different subsectors 
and education and training institutions and external stake- 
holders, particularly the private sector, on skills needs and 
relevant responses. 


Learning outcomes have also the potential to act as tools 
for cooperation and communication between teaching 
staff (for example TVET teachers and enterprise trainers in 
the context of apprenticeship), between education institu- 
tions (for example TVET and higher education institutions), 
and between education and the labour market institutions 
(for example career guidance and employment advisors). 


1.4.4. Learning outcomes for education and training evaluation 
and accountability 

The emergence of this ‘common language’ can support 
orienting existing resources to support capacity build- 
ing for evaluating learning outcomes at national, regional 
and global scale. Standardised learning assessments are 
increasingly used for holding teachers, education and 
training institution leaders, institutions or entire systems 
accountable. A network is already in place to move this for 
ward. The Global alliance to monitor learning (GAML), es- 
tablished by the UNESCO Institute for Statistics, provides 
methodological solutions to develop a new indicator on 
learning and to set standards for good practices on learn- 
ing assessments. Member States and technical experts 
from around the world have come together to develop an 
innovative but pragmatic approach that recognises diver 
sity while yielding internationally comparable measures of 
learning (Benveniste and Montoya, 2016). 


1.5. Conclusions 


This chapter started with presentation of the SDG agenda 
and the place of qualifications in SDG 4 on education. It 
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emphasised the shift of policy objectives of NOFs from 
a purely economic perspective to a holistic development 
paradigm. We suggested that the way learning outcomes 
are considered in education and training is of key impor 
tance to the achievement of SDG 4. In this context, SDG 4 
has the potential both to reinforce the use of learning out- 
comes as ‘common language’ of education and training 
systems in a lifelong learning perspective and to connect 
the analysis of NOF systems with intended sustainable de- 
velopment outcomes. 


Since the success of the learning-outcomes-centred ap- 
proach depends on the capacity of decision-makers and 
practitioners, there is a clear need for international coop- 
eration in this important area. The World reference levels 
platform mentioned in this inventory (see Chapter 8) could 
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NATIONAL QUALIFICATIONS FRAMEWORKS 


Michael Graham and Arjen Deij, ETF 


Summary 


As many as 154 countries worldwide are developing na- 
tional qualifications frameworks (NOFs). However, many 
countries have difficulties in moving from conceptual de- 
velopment and official NOF adoption to implementation: 
frameworks with modern, outcomes-based qualifications. 
Since the previous inventory was published in 2015, few 
new NOFs have been developed and the number of coun- 
tries that have moved towards full implementation is limit- 
ed. In ETF partner countries, most NOFs are still only par 
tially implemented. We examine why countries are blocked 
and suggest how they might speed up implementation. 
We argue that to put the NOF in place, you also need 
to look at the qualification system. We identify four ele- 
ments common to all qualification systems in the partner 
countries, which shape and determine how well a system 
works: legislation, stakeholder involvement, institutions, 
and quality assurance. Partner country cases are exam- 
ined, and we set out our analysis of what makes targeted 
and proportionate legislation, propose useful stakeholder 
platforms, identify the types of roles and functions specific 
institutions should perform and what set of quality assur 
ance arrangements ensure quality qualifications. Then we 
set out some findings. 


Keywords: national qualifications framework (NQF); 
system; legislation; stakeholders; institutions; quality 
assurance (QA) 


2.1. Introduction: 
countries at the crossroads 


The European Training Foundation (ETF) is an EU agency 
which works with 29 countries in the Western Balkans 
and Turkey, Eastern Europe and Central Asia, and the Mid- 
dle East and North Africa. This third edition of the Global 
NOF inventory finds that most of these 29 countries have 
progressed to the implementation stages of their NOF. 
But while most have an NOF, true implementation — de- 
livering new qualifications and in significant numbers — 
eludes the majority. 


After an initial surge around 2010-12, momentum Is slow- 
ing. Most countries have a consensus to proceed with an 
NOF, have laws, and have allocated roles to institutions. 
Some have implementation plans, designed quality assur 
ance systems and have developed criteria for qualifications 
structure and content. A vanguard group has adopted 
methodologies for vetting qualifications to enter the na- 
tional register and placed a range of new qualifications, 
including different types such as adult qualifications, in its 
NOF. In some countries, however, little has changed since 
the respective law was adopted. 


Most countries are somewhere in the middle: their NOFs 
are partially in place. While it is progress that most have 
moved from concepts to implementation, some express 
frustration that they are not more advanced, a position 
which risks discouraging further work. 


The ETF sees that NOFs can be useful tools in education 
and training reform. NOFs as classification systems bring 
order to the qualifications landscape; they are established 
as tools in the partner countries, in most cases, as vital to 
producing relevant lifelong learning qualifications. NOFs, 
crucially, establish outcomes as the conceptual paradigm 
for level descriptors, qualification and curricula design. 
Frameworks are also the entry point and international ref- 
erence point for comparing qualifications across borders. 


If countries agree, why are they experiencing difficulty in 
moving to the implementation stages? They have plans, 
laws and have produced some standards and new qual- 
ifications to equip their citizens for modern economies; 
these imply many jobs in an individual lifetime. They can 
write learning outcomes for the qualifications or drafting 
the NOF level descriptors. They either have this technical 
expertise or can hire it. 


It seems their tougher challenge is in making the NOF 
work within the wider education and training systems and 
connecting it to society and the economy. The hard part 
seems to be building a sustainable infrastructure, making 
the challenge less technical than social and institutional. 
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The ETF therefore, identifies the challenge for the coun- 
tries as getting organised. 


We cannot understand why countries are blocked in imple- 
menting an NOF until we look deeper than the framework. 
NOFs alone, are not enough: the goal is, simply put, to 
create good qualifications. 


That requires a functioning qualification system. Let us 
make a simple and clear distinction: an NOF is an instru- 
ment used to classify qualifications in a hierarchy of lev- 
els, each level defined by a set of descriptors indicating 
the learning outcomes at that level. A qualification system 
is all the arrangements — schools, institutions, stakehold- 
ers, laws, quality assurance measures and the framework 
of qualifications — which ultimately deliver better qualifica- 
tions for employability and lifelong learning. 


In a recent article, Stephanie Allais (2011) criticised 
NOFs as ineffective and suggested that the ETF endors- 
es NOFs. Our concern is that our 29 partner countries 
produce good qualifications. Individuals need quality 
and reliable qualifications to demonstrate their compe- 
tence for work in an era when they change jobs more 
frequently, and qualifications which certify a broader 
range of skills. Qualifications link work to training and 
education, and so provide an understandable common 
language. NOFs are a tool to contribute to quality, reli- 
ability, transparency and comparability. In the countries 
where we work, we see evidence that the NOFs do con- 
tribute to the quality of qualifications: this is notable in 
Turkey, Kosovo, Montenegro and other countries which 
are advanced. 


NOFs and qualifications are social constructs, having 
meaning only if society uses them. It follows that coun- 
tries face social and institutional challenges in making an 
NOF work, that extend beyond the NOF technical speci- 
ications. To have impact, NOFs need to be part of func- 
tioning qualification systems. Therefore, in this chapter 
we place NOFs -— the focus of this global inventory — in 
the context of the system around them. We look at the 
institutions, actors and processes involved and how regu- 
lation and legislation, stakeholder interaction, institutional 
arrangements and quality assurance mechanisms contrib- 
ute to improved qualifications. 


This chapter sets out to establish why countries some- 
times stall in moving from planning to execution, proposes 
an analysis of the necessary components of a qualification 
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system they need to prioritise to restart reform, and identi- 
fies commonalities found in successful systems. 


2.2. Four components of a qualification system 


Qualifications systems can only work effectively if their 
different organisational elements function together. The 
ETF is not an academic body or research institution; in- 
stead we advise the 29 countries on reform. Our obser 
vation is that integrated and coherent systems produce 
better qualifications. 


We have identified four foundation components of a qual- 
ification system, which are common to all our partner 
countries, regardless of progress, economic diversity or 
any other specific local conditions (ETF, 2016). These are: 


(a) the legal and regulatory framework; 
(b) effective stakeholder dialogue; 

(c) institutional arrangements; 

(d) quality assurance systems. 


Laws stipulate functions of the NOF and criteria for 
qualifications and allocate tasks to specified institu- 
tions. Effective dialogue with stakeholders is about 
relevance and ownership of qualifications and their ac- 
ceptance in the education system and labour market. 
Institutions are needed to ensure professional process- 
es in developing standards and qualifications, coordi- 
nating stakeholders, developing and maintaining tools 
and methodologies such as registers, and for levelling 
qualifications in the framework and quality assurance. 
Quality assurance provides confidence in the qualifica- 
tions and those who hold them. 


We now take each of these four elements in turn. 


2.2.1. Legislation 

Most partner countries start any public policy with a law, so 
getting the legislative and regulatory framework for the qual- 
ification system right is vital. Countries, broadly, use laws 
to regulate things they want to change. Reforming qualifi- 
cations involves many issues, including developing qualifi- 
cations based on learning outcomes, involving employers, 
quality assurance measures, and setting up an NOF. 


Laws should support production of new qualifications but, 
too often, in practice they obstruct this aim. We looked at 
legislation in 11 EU countries and In our partner countries, 
uncovering a wide range of primary and secondary legisla- 
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tion applying to NOFs, establishing institutions, or creating 
new qualifications. 


We found it helpful to examine legislation for qualification 
systems in two dimensions: substance and process. The 
former concerns the laws, regulations, and their scope and 
intentions. The latter looks at how law is conceived — its 
context in other strategies and laws — how it will be linked 
to these; how coherence and consistency in principle and 
objective is achieved between qualification systems, the 
wider education and training system and regulations apply- 
ing to the labour market. 


We found that reform aimed at better qualifications re- 
quires eight key parts of legislation. These can be subdi- 
vided into Parts 1 to 3 - which regulate the principles and 
institutional foundations of qualification systems — and 
parts 4 to 8- which regulate more organisational aspects 
of qualification systems such as the NOF, stakeholder in- 
volvement, quality assurance, validation of non-formal 
learning and recognition of qualifications. 


Key part 1: purposes and principles 


Laws need a purpose, which sets out what the law will 
introduce or change. Laws need to be based on prin- 
ciples, which describe the context of the law and give 
the rationale for its introduction; they explain why the 
law is required. A law on an NOF will have as its pur 
pose to regulate the structure, as in levels and descrip- 
tors. Its purpose might be to promote lifelong learning 
and improve relevance. 


Key part 2: regulating institutional arrangements 


Laws also establish the arrangements that regulate 
the roles and responsibilities of the competent bod- 
ies, including specifying which institutions must act to 
implement the law and achieve its purposes. Kosovo's 
2008 Law on Qualifications stipulates matters such 
as the national qualifications authority's (NOA) status 
and relationship with its Governing Board, and its de- 
cision-making procedures. The NOAs functions are de- 
tailed: in regulating qualifications, maintaining the OF 
regulating award of qualifications. 
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Key part 3: regulating stakeholder involvement 


Laws can also regulate or formalise the role of stake- 
holders in implementing elements of a qualification 
system. Turkey's implementing regulation on sector 
councils defines procedures for setting up sector 
committees, their governance and work procedures 
and their functions. It foresees sector committees 
as collegial multipartite entities providing counsel- 
ling, and executing review and quality assessment 
of occupational standards. Sector committees will 
provide the expertise and feedback of sectoral 
stakeholders in a more centrally organised national 
system of qualifications. 


Conversely, legislation has been known to erect bar 
riers to stakeholders. In Tunisia, pre-revolution legisla- 
tion hinders, at least formally, engagement of employ- 
ers and trade unions in continuing vocational training. 


Key part 4: regulating development of qualifications 


Laws should also address the qualifications them- 
selves: their design, development and quality and 
delivery. Primary laws on NOFs are often used to 
specify the principles of qualification structure and 
content, such as basis in learning outcomes or re- 
quirement to use occupational standards as basis 
of VET qualifications. (However, it can be the case 
that design and award of qualifications is prescribed 
or provided for in other education and training legis- 
lation). By contrast, secondary legislation regulates 
methodologies, provides for quality assurance pro- 
cedures such as specifying in detail criteria qualifica- 
tions must meet to be entered in the national regis- 
ter or levelled in the framework. 


Key part 5: specifying key issues for implementing 
the NOF 


Some countries have a separate NOF law, while for 
others, such as Kosovo, the NOF is part of a broader 
law targeting other areas of the qualification system. 
Increasingly, this latter route is the most favoured; for 
example, Albania’s new laws on HE and VET relate to 
the Albanian qualifications framework. 
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Issues related to the NOF usually included in legislation are: 

e scope: types of qualification included in the NOF; 

e structure: levels and level descriptors; 

e¢ management: both of the NOF itself and the imple- 
menting institutions; 

e aregister or database of qualifications; 

e relationship with other instruments, such as existing 
classifiers (national classification systems of occupa- 
tions), economic sectors, education programmes; 

e access to qualifications, to the horizontal or vertical 
progress between qualifications and qualification 
levels, and to the transfer of credits; 

e learning outcomes as the basis for qualifications; 

e quality assurance both of the qualifications in the 
NOF and the framework itself; 

e validation of non-formal and informal learning; 

e EOF linking to the wider European and internation- 
al dimensions. 


Key part 6: regulating quality assurance 


Laws regulate quality of qualification standards, as- 
sessment, and certification; the bodies responsible 
for quality assurance of qualifications, and the coor 
dination between these bodies; and procedures and 
criteria for the inclusion of qualifications in an NOF da- 
tabase or register. 


For example, Albania's NOF Law specifies that ‘a qual 
ification is awarded when a competent body decides, 
by means of a quality assurance assessment process, 
that the individual has reached the specified standards’. 


Key part 7: regulating validation of non-formal and in- 
formal learning (VNFIL) 


An NOF of comparable qualifications based on learning 
outcomes, can support alternative access to qualifica- 
tions and their recognition. 


In the EU, NOFs and VNFIL systems are closely 
connected, a relationship which is strengthened by 
the 2012 EU recommendation on VNFIL, as its pro- 
visions explicitly link VNFIL implementation to coun- 
tries’ NOFs. 
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Key part 8: regulating recognition of qualifications 


The terms ‘validation’ and ‘recognition’ are often 
used interchangeably, though they have different 
meanings. Validation is confirmation of an individual's 
knowledge, skills, and competences; recognition is 
the external recognition of a qualification, the certifi- 
cate issued to that individual, including recognition of 
overseas qualifications. 


The legislative process: drafting and developing legislation 
The second broad area of law in qualification systems Is 
process, the business of where countries should start, 
aligning old and new legislation, and linking framework leg- 
islation to more specific regulation. Countries have found 
they need to ensure consistency between laws aimed at 
the qualification systems and those on related policies, 
particularly the broader education and training systems 
and the labour market. 


Countries, from our observations, do best when they start 
legislative processes only after first agreeing a strategy 
for qualifications reform, itself the product of broad stake- 
holder consensus. The law should address the aims of the 
strategy: to build more trust in qualifications; aid matching 
of qualifications supply from education and demand from 
the labour market; and identify measures to future-proof 
qualifications in emerging occupations. 


The second issue to decide is where this new law sits with 
existing laws. Every country has laws already, so starting 
from a blank slate is not really possible. Inevitably, this 
means old and new laws coexist for the same targeted 
area. Qualifications are often part of both education leg- 
islation and labour legislation. There are two main options 
when aligning old and new laws: either adapt existing laws 
by amending them and adding some secondary regula- 
tions; or developing a new legislative framework. This re- 
quires first a mapping of existing legislation. 


For example, Albania has recently revised the NOF law first 
adopted in 2010 but it was never properly implemented. 
Other relevant laws taken into account to create consist- 
ency with the revision of the NOF Law are the new Law 
on Higher Education (adopted in 2015); the new VET Law 


GLOBAL INVENTORY OF NATIONAL AND 
REGIONAL QUALIFICATIONS FRAMEWORKS 


(adopted in 2017); a new Law on Crafts (prepared in 2016); 
and the new Labour Code (revised in 2015) ('). 


Primary and secondary legislation 

In qualification systems, countries use primary law to 
set general principles, as with the Primary Law on the 
Vocational Qualifications Authority (VOA), 2006 in Tur 
key (*); these are adopted by the parliament. Secondary 
legislation defines detailed provisions, usually a decree 
or similar issued by the Ministry of Labour, for example 
implementing regulations on criteria for developing occu- 
pational standards. 


Another case is Kosovo. Its 2008 law on qualifications is 
primary legislation, has broad scope, covering the estab- 
lishment of a national qualification system based on an 
NQF; the law is supplemented by a range of secondary 
legislation regulating issues such as validation of non-for 
mal learning and development of occupational standards. 


In most countries, NOFs are being legislated by decrees; 
examples are Azerbaijan, Moldova and Ukraine. In con- 
trast, Albania, and the Former Yugoslav Republic of Mace- 
donia adopted their NOFs via primary laws. 


Balancing tight and loose legislation 

How prescriptive or liberal countries are in legislation is 
partly a matter of tradition. Most of the 29 partner coun- 
tries have a mix of some laws that are more prescriptive 
in phrasing and intention and some legislation that is more 
empowering. Broadly speaking, most partner countries 
are heirs to the more directive, as opposed to permissive, 
law-making tradition. 


By contrast, in English-speaking countries precedent and 
practice weigh more than legislation and regulation is lim- 
ited compared to most EU countries. NOFs often perform 
a regulatory function of a market of qualifications. 


Civil code-led legal systems typically apply tighter regula- 
tion. High-value qualifications here are the product of con- 
sensus and cooperation among actors, meeting compulso- 
ry criteria. Qualifications frameworks in such countries do 
not regulate a market of qualifications but establish prin- 
ciples for reliable qualifications. For example, in Germany, 


(‘) See national case of Albania in Global inventory of regional and 
national qualifications frameworks. Vol. 2. 

(2), See national case of Turkey in Global inventory of regional and na- 
tional qualifications frameworks. Vol. 2. 
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the Ausbildungsordnungen have the force of law and are 
agreed between social partners and the Federal Govern- 
ment. Ensuring that learners have access to well-defined 
and agreed broad-based qualifications means they will be 
better equipped for employability. 


In the partner countries, central governments tightly reg- 
ulate the roles of the actors in their qualification systems. 
They have State-regulated qualification systems, but social 
dialogue is weak. Countries lack experience with markets 
as well as consensus structures, because, in the past, the 
State decided what was good for everybody. 


Stakeholder input to laws 

In EU neighbourhood countries, Ministries of Education 
typically lead drafting of new laws or revision of existing 
legislation. But defining the content of a law applying to 
qualifications cannot be left only to lawyers and officials. 
Stakeholders need to inform legislation. If they are active 
in the qualification system strategy, they can generally ex 
pect to influence the law. 


In some cases, the private sector has proposed the law. 
This happened in Ukraine, where employers pushed for an 
NOF, resulting in the NOF law of 2011. Employers are also 
central to current development of a new National Frame- 
work Law on Education. Here, as in other partner coun- 
tries, employers’ priorities are adult learning and relevant 
qualifications, while the Ministry of Education looks more 
at the needs of education institutions and broader educa- 
tion for citizens. 


2.2.2. Stakeholder involvement 

Employers in partner countries often lack confidence in 
the qualifications candidates offer, which they perceive 
as irrelevant or not understandable. Some stakeholders 
in our partner countries say that skills matter rather than 
qualifications. But qualifications are a proxy for skills, so 
countries prioritise them in VET reform. Plus, qualifications 
without stakeholder engagement lack credibility. 


While the term stakeholder is common, we should dis- 
tinguish it from its synonym, actor. Stakeholders are peo- 
ple, groups or entities with a specific role and a vested 
interest in the implementation, in this case, of qualification 
system-related policies. Actors are authorised agents who 
can exercise agency (the ability to effect change) in a con- 
text. Not all stakeholders have this agency so not all of 
them are actors. By contrast, all actors are stakeholders. 


26 


VOLUME | — THEMATIC CHAPTERS 


Visually, we can think of interest and influence in this schema. 


MINIMAL EFFORT 


Individual learners 


Individual employers 


LOW 


Employment services 


Media channels 


LEVEL OF INTEREST 


KEEP INFORMED 
International organisations 
Public TVET providers 


Methodological centres 


Consultants/experts 


Professional associations 


Source: adapted from Johnson and Scholes (1999), Exploring Corporate Strategy. 


Identifying and mapping stakeholders 

Various typologies of stakeholders exist, but we have dis- 
tilled them to five distinct groups: public authorities and 
the State, such as ministries and agencies; industry bod- 
ies and chambers of crafts and commerce; education and 
training providers; learners; and international donors, e.g. 
Deutsche Gesellschaft fur Internationale Zusammenarbeit 
(GIZ), the British Council, and NGOs. 


However, It is usually the case that the lead body in the 
NOF (normally, at least initially, a ministry, then perhaps a 
specialised autonomous agency — see Section 2.2.3) engag- 
es with these various stakeholders with variable intensity, 
rather than uniformly. The lead will prioritise, from what we 
have observed, on relative levels of interest and influence 
attributed to the respective identified stakeholder. 


How to engage stakeholders, and in what format, can be 
established via a fourstage process: 


(a) identifying: listing and mapping relevant groups, organi- 
sations and people; 

(b) analysing: understanding stakeholder 
and interests; 

(c) mapping: visualising relationships, mapped to objectives 
and to other stakeholders; 


perspectives 


(d) prioritising: identifying issues and ranking stakeholder 
relevance by likely impact. 


Coordinating stakeholders 

Given that responsibility for quality of qualifications rests 
with the government, the State normally initiates stake- 
holder mobilisation. But, too often, the mind set in min- 
istries is that stakeholders are interest groups to be man- 
aged, rather than forming part of acommon public interest. 
In Azerbaijan, development of a common interministerial 
approach to the NOF came first, so that contacts with 
stakeholders already started from a positive, common po- 
sition. Designating one ministry as lead will help dialogue 
with stakeholders, who often complain of the complexity 
of State institutions or weak links between them. 


Dialogue: platforms, types and topics 

Stakeholders from various organisations convene to 
share a dialogue platform. Dialogue is about agreeing 
strategy: implementation is the technical work resulting 
from the strategy’s agreed actions. While stakeholders 
most often discuss and advise and institutions imple- 
ment, this distinction is rather blurred in practice. In 
countries such as Germany and Turkey, where employers 
strongly engage in the VET system, stakeholders have an 
implementing role. 
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One obstacle in getting stakeholder buy-in in partner coun- 
tries is the shortage of major industrial companies with 
capacity for expert engagement with education or who 
can establish business representative groups as partners 
in addressing education and training issues. Most partner 
countries depend for their economic growth on numerous 
SMEs, not several big companies. As in any country, SMEs 
lack staff and time to delegate their people to participate in 
dialogue with government. 


However, in some cases, partner countries bridge this gap 
by setting up sector representative bodies or turning to 
chambers of commerce, generally broader in the types 
of business than equivalents in EU countries. Kosovo's 
Chamber of Commerce sits on the National Qualifications’ 
Authority board. 


Formal platforms exist in several ETF partner countries and 
take various forms. Turkey has sector committees, which 
are tripartite structures with State, employer and employ- 
ee representatives; their establishment, duties and opera- 
tion are regulated by law; 23 are currently operational. Sec- 
tor committees perform tasks related to development and 
maintenance of occupational standards and qualifications 
for their sector. This is an example of a stakeholder body 
discharging an implementation function. 


Bosnia and Herzegovina has an intersectoral commission 
for its NOF which has both political and geographic com- 
ponents. It comprises the State-level Ministry of Civil Af- 
fairs plus representatives of the different communities in 
the country: stakeholder elements such as employers and 
trades unions; experts from higher education institutions; 
plus experts from specialised agencies such as those 
managing quality assurance or VET systems. 


In other cases, there may be only informal platforms, as 
in Ukraine, where dialogue continues among stakeholders 
who were involved in the previous NOF working group ac- 
tivities, and Azerbaijan, where ETF-chaired dialogue sought 
to increase social partner representation in the official NOF 
working group. 


Moldova has both VET and HE stakeholders on its sectoral 
committees, working with public institutions and donors. 
Sectoral committees have a role in identifying which occu- 
pational standards and qualifications should be developed 
for the sector; this can be considered a form of sectoral di- 
alogue. Their responsibility to develop occupational stand- 
ards is an implementing task. Developing a partnership 
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around a central qualifications agency to coordinate stake- 
holders is one option to bolster engagement efforts led 
by ministries. A coordinating agency can also be a neutral 
meeting place for stakeholders. 


While informal dialogue may be useful in allowing partici- 
pants to speak more freely than in more formal settings, 
without institutionalisation in the shape of sector commit- 
tees or formal membership (for example, of NOF working 
groups), they risk having only marginal influence on the 
reform process. 


There are various potential topics for dialogue: analysing 
problems in the current qualification system; developing 
an NOF implementation plan; formulating reform objec- 
tives; identifying need for new qualifications, reviewing 
institutional capacities and arrangements; and deciding on 
changes to qualifications. 


We see four formats for dialogue with stakeholders, using 
their feedback to shape reform: informative, consultative, 
cooperative, and decisional. In informative dialogue, the 
public authorities only inform stakeholders about deci- 
sions already taken on qualifications; arguably, this is not 
dialogue at all. Consultative dialogue means authorities 
consult stakeholders and their feedback may or may not 
shape ministry decisions. Cooperative dialogue goes fur 
ther, so that stakeholders sit on decision-making bodies 
and share in those decisions. Decisional dialogue hands 
decisions to stakeholders themselves, prioritising in the 
defined area where they lead. 


Countries can use one or all of these formats, though in 
less inclusive social and political systems only the first 
format will be used. The format is usually influenced by 
the dialogue topic, so that, in practice, all four formats are 
used variably. 


Stakeholders in the qualification development process 

Three distinct stages can be identified in stakeholder en- 
gagement in the qualifications development process: poll 
cy, design and implementation. 


The policy stage is concerned with the functions, proce- 
dures, and regulatory context of qualifications, as well as 
funding and support mechanisms. For example, there may 
be a national committee of various categories of stakehold- 
er, some with donor participation. In Morocco, while em- 
ployers worked with the State, trades unions and NGOs 
were not engaged. 
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The design stage is about establishing learning outcomes 
and agreeing assessment standards for qualifications. 


The implementation stage concerns actual use of qualifi- 
cations and their application via teaching and curricula and 
assessment. Stakeholders such as enterprises, may have 
an assessment role and an implementation function, a key 
element in the German dual system. 


2.2.3. Institutions: moving from bureaucracy to service 

All countries have institutions which manage qualification 
systems and exercise functions of design, implementa- 
tion, coordination and regulation. Where countries reform 
their qualifications systems, typically using an NOF as the 
main instrument, these functions and roles, and the rela- 
tions between institutions, will change. 


Institutions in partner countries struggle to manage this 
transition from traditional qualification systems (dominat- 
ed by the State and producing mainly school-leaver VET 
qualifications, issued under the aegis of the ministry of Ed- 
ucation), to more of a lifelong learning system of diverse 
providers, aimed at young people, adults, professionals 
and job-changers. 


These different qualifications may be based on differ- 
ing principles and issued by different bodies, not under 
the control of the Ministry of Education. Bringing qual- 
ifications offered by diverse providers into a common 
framework based on learning outcomes should make 
such qualifications comparable. Countries are seek- 
ing to regulate the quality of qualifications. NOFs can 
bring some order by linking these qualifications but this 
alone is not enough. Rules, institutions and organised 
stakeholder involvement, to ensure quality and linkag- 
es, are also necessary. However, countries need also 
to ensure that these measures do not tip over into 
more bureaucracy, rather than offering a useful service 
to stakeholders. 


Institutions differ from stakeholders in being specialised in 
qualification systems or exercising an allocated function, 
such as developing standards, conducting assessments or 
coordinating stakeholders. 
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Institutions’ roles and functions in a qualification system 
Institutions generally exercise design and implementation 
functions and transversal functions. This simplifies, as 
there is conceptual overlap. 


There are various design and implementation functions: 
developing models for standards and qualifications, proce- 
dures for assessment, procedures for validation of non-for 
mal and informal learning, and systems for recognition of 
learning outcomes; developing and maintaining occupa- 
tional standards and qualifications; registering qualifica- 
tions in aregister; assessment and certification of learning 
outcomes and recognising learning outcomes. Transversal 
functions concern coordination and system review, com- 
munication, career information and guidance and quality 
assurance and regulation. 


Many of the processes involved in managing and imple- 
menting a qualification system apply regardless of an NOF 
but a framework should act as a tool to link these functions 
and roles. 


Types of institution 

The complexity and range of which institutions or bodies 
discharge which of the above functions is considerable. 
Among our partner countries, we cannot see a standard- 
ised, common, set of institutional arrangements. Country 
practice varies widely, and categorisation is further com- 
plicated by overlaps in institutional roles, so that even de- 
tailed study cannot point to the exercise of mutually ex 
clusive functions (or definitive solutions). Systems evolve, 
and so do arrangements. In some cases, countries devolve 
functions from ministries to specialised VET agencies or 
qualifications authorities. This process is not always with- 
out friction, as ministries may resent loss of authority and 
indeed budget. 


However, we can identify broad categories of institutions, 
which we hope can aid understanding: the coordinating 
institution; sector skills councils or sector committees; 
bodies conducting assessment and certification; and insti- 
tutions which regulate for quality of qualifications. 


Coordinating institutions 

Usually, NOFs are initiated and, in their early days, run, by 
ministries of education, sometimes of labour. At this ad 
hoc stage, governance is often by committee, or several 
technical committees and working groups. Institutionally, 
a critical stage is reached when committees pass the NOF 
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ball to the council or ministry office or agency to put the 
NOF in place. 


In some partner countries, broad coordinating councils 
seek to manage implementation. However, when these 
are too big, they can meet too infrequently to make deci- 
sions. And because they are interministerial, their access 
to budgets can be limited. 


A small executive committee can work well. Turkey has 
benefited from Its agile five-member executive board of Its 
Vocational Qualifications Authority (VOA), which collective- 
ly takes major decisions. It is linked, via the Turkish qualifi- 
cations framework, to the TOF Higher Council, which is in 
charge of decision-making for the qualifications framework 
(QF) overall. The TOF Council advises rather than decides 
and can delegate technical issues to relevant experts. 
Kosovo's governing board is small and similarly efficient at 
decision-making, accelerating NOF implementation. 


The ETF's experience over several years has been that the 
countries with dedicated authorities do best and go quick- 
est. These authorities — autonomous agencies composed 
of professional, expert staff — exercise delegated real pow- 
er. It matters less if their numbers are small, as expertise 
or other support can be subcontracted or cooperation with 
donors sourced, but political support is critical. Turkey's 
VOA Is effective as its establishment was stakeholderin- 
fluenced and it is accountable to a representative board. 


Common institutional roles performed by these new bod- 
ies include NOF coordination and communication, devel- 
oping occupational standards (or commissioning them), 
developing and managing the qualifications register, and 
conducting OA functions. 


Institutions regulating qualifications quality 

Coordinating bodies often have formalised roles in quali- 
ty-assuring qualifications. QA and regulation functions in- 
clude accrediting and registering qualifications, accrediting 
providers, accrediting awarding bodies, developing guide- 
lines for internal OA processes, defining national indica- 
tors, conducting external validation of summative assess- 
ment and reviewing the OA system as a whole. 


VOA Turkey, for example, accredits and registers quali- 
fications; the National Centre for Quality Enhancement 
in Education accredits providers in Georgia, while the 
National Qualifications Authority in Kosovo accredits 
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awarding bodies and develops criteria for qualifications 
to enter the NOF. 


Sector skills councils and sector committees 

Many countries have set up sector skills councils to iden- 
tify skills needs, to set principles and priorities for qual- 
ifications development, to develop occupational stand- 
ards, develop and review qualifications, and to assess 
candidates. Sector skills councils or sectoral committees 
require labour market experts. When institutionalised and 
staffed by experts, such bodies are the best guarantee 
of vocational qualification relevance and of labour market 
acceptance of qualifications. 


Sector skills councils are well established in Turkey, act- 
ing under the VOA umbrella. Tripartite committees define 
needed occupational standards and qualifications, and de- 
velopment of the standards is undertaken by bodies which 
sign agreements with the VOA. Funding for these is from 
sector bodies, which volunteer to develop the standards; 
the sector committee reviews the standards before valida- 
tion by the VOA. 


Russia and Ukraine are also establishing councils, but from 
a different starting point. Here, the initiatives to establish 
the councils came from the employers, rather than a min- 
istry. Their main outputs so far have been occupational 
standards. Russia's National Qualification Development 
Agency (NARK) and Ukraine's Institute of Professional 
Qualifications also aspire to establish independent assess- 
ment systems to certify workers against the standards al- 
ready developed. 


New institutions for assessing and certifying 

learning outcomes 

New qualifications need to be trusted. Partner countries 
are increasingly turning to external assessment, instead of 
purely school-based methods. 


In Azerbaijan, the State Examination Centre acts as eter 
nal examiner for HE admissions, and is now applying this 
function to post-secondary VET colleges; it also deals 
with the final examinations for secondary education. Cru- 
cially, such bodies are independent from ministries. The 
State Examination Centre develops assessment strate- 
gies, identifies and recruits assessors, ensures that sum- 
mative assessment is based on the same standards, and 
ensures that results are secure so certification reaches 
successful candidates. 
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Montenegro's Examination Centre also leads in organising 
validation of non-formal learning. 


Turkey has a sophisticated system for adult learner assess- 
ment. Its 60 VOC test centres (formally titled authorised 
certification bodies) assess and certify adult skills against 
349 national vocational qualifications. These centres are 
authorised by the Turkish Vocational Qualifications Author 
ity, and assess against approved occupational standards. 
To date, the VOC test centres have issued approximately 
115 000 certificates. Their institutional roles include trans- 
lating qualifications standards into assessment tools, de- 
veloping assessment strategies, identifying assessors, 
ensuring security of assessment results, and offering guid- 
ance to unsuccessful candidates. 


New awarding bodies 

Private awarding organisations are found in some partner 
countries, responding to the diversifying market of qualifi- 
cations. Turkey, Kosovo and Montenegro all accredit such 
organisations, while in Ukraine and Russia the potential is 
significant given the strength of industry. 


Combining or separating roles 

The dividing lines between sectoral bodies, awarding bod- 
ies, independent assessment bodies, coordinating institu- 
tions, and OA bodies are not clearcut. Some both support 
quality of provision and ensure the quality of qualifications. 
There are clear advantages to these apparent grey areas 
and synergies in bringing different executive functions 
under one umbrella, including accelerating impact of new 
qualifications on programmes. 


Although combining functions under one roof may blur 
boundaries among subsectors, we see more risk in hard 
separation into silos, which usually results in competing 
institutions, unclear remits, and inefficiency in sharing 
resources. Examples include where quality assurance 
bodies and qualifications bodies operate separately in 
HE, VET, and general education, or between ministries of 
education and labour. 


Conversely, countries are seeing the gain in merging OA 
functions across VET and HE, matching and transparently 
implementing the Bologna and EQF (where applicable) 
requirements in quality assurance to support lifelong 
learning. Examples include the National Centre for Edu- 
cation Quality Enhancement (NCEQE) in Georgia. In Mol- 
dova, the authorities are establishing a new OA Agency 
for both HE and VET. 
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2.2.4. Quality assurance of qualifications 

Quality assuring qualifications links laws, stakeholders and 
institutions as legislation prescribes criteria, stakeholders 
bring relevance and institutions perform the quality assur 
ance functions. 


Quality assurance is especially critical at the level where 
qualifications are delivered: the qualifications and the qual- 
ified individual. Concern here is with standards, assess- 
ment and certification. Our understanding and use of these 
terms is as follows: standards define the requirements 
for awarding qualifications; assessment is the process of 
verifying whether someone meets the learning outcomes 
specified; and certification is the process of awarding the 
qualification to the candidate. 


Qualification standards, assessment and certification 

In most partner countries, one national body designs 
and specifies assessment methods, and assessment 
is centralised or delegated to providers, so that qualifi- 
cations are awarded against centrally designed exams. 
Emphasis is on control, via inspection and a culture of 
compliance. OA is still mainly focused on providers, less 
so on qualifications or ensuring the learner meets the 
outcomes. Both are required and, in partner countries, 
quality assurance of schools, teaching and curricula is 
necessary for modernisation. 


Countries are experiencing an increase in the number and 
range of programmes and qualifications, including those 
offered by private providers or NGOs; occupational stand- 
ards are generated by donors in high numbers. They are 
looking more for external OA, via VET agencies or OA bod- 
ies, and seek more relevance in the qualifications devel- 
opment stage by engaging stakeholders, such as through 
sector skills councils. 


The advent of NOFs has provided a means to review OA 
arrangements and mechanisms to initiate reform in this 
area. In most partner countries, these changes are oc- 
curring in both VET and HE, influenced externally by the 
EOF and Bologna respectively; OA is also being pushed 
up the agenda. To some degree, a shift from the usual 
focus on inputs and providers to outputs such as qualifi- 
cations and what learners can do with their qualifications 
is taking place. 


More progress is observable in developing the standards 
behind the qualifications than in awarding qualifications, as 
in assessment and certification. 


GLOBAL INVENTORY OF NATIONAL AND 
REGIONAL QUALIFICATIONS FRAMEWORKS 


Quality qualifications: the quality chain 

OA for qualifications comprises two broad processes: en- 
suring that qualifications have value; and that the people 
who are certified meet the conditions of the qualification 
award. Necessary elements are the standards behind the 
qualifications, placing qualifications in the NOF learning 
pathways, assessment and certification. 


Standards behind qualifications 

Countries generally use two key criteria: labour market in- 
put, via the type of stakeholder dialogue process covered 
earlier, and learning outcomes. In this sense, who is as im- 
portant as what in determining qualification quality. Qualifi- 
cations without stakeholder input will not command trust. 
Labour market representation in processes and methodol- 
ogies for developing qualifications is required by countries 
such as Kosovo and Turkey. 


Outcomes-based standards allow assessment of com- 
petence. The Turkish OF and many other cases have ad- 
vanced most in this direction in VET qualifications. The 
TOF distinguishes between qualifications in the national 
education and training system, those in the national vo- 
cational qualifications (NVQ) system and those awarded 
by other bodies. NVQs closely follow a similar pattern of 
units, derived from occupational standards; these are of- 
ten one-to-one, standard to qualification. Each unit is de- 
scribed in learning outcomes and performance criteria and 
contains assessment guidance. 


Placing qualifications in a register 

Levelling individual qualifications in the framework is a key 
quality process, both internally, to encourage trust in the 
qualifications, and internationally, for comparison. NOFs 
drive quality improvement in qualifications by stipulating 
criteria in design, relevance and assessment. Criteria vary 
in partner countries, depending on the type of qualifica- 
tion, for example between those typically offered in initial 
VET and those aimed at jobseekers. Kosovo has unit-based 
qualifications in areas such as welding for adults undergo- 
ing reskilling. Non-formal qualifications — not part of formal, 
usually State, provision — may be regulated by differing cri- 
teria, and special levelling procedures. 


In the Former Yugoslav Republic of Macedonia, the ETF 
worked with national authorities to conduct an invento- 
ry of existing VET qualifications, then analysed a sample 
against the NOF level descriptors to test their ‘true’ level 
in the NOF. The analysis compared outcomes of individu- 
al qualifications with level descriptors, relevance and de- 
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mand on the labour market, the standards supporting the 
qualification and engagement of social partners in devel- 
oping the qualification. 


Learning pathways 

Learning pathways can differ between qualifications. 
Countries are increasingly looking at certifying learn- 
ers against standards rather than simply awarding cer 
tificates after completion of traditional schooling. While 
some partner countries such as Kosovo, Montenegro and 
Turkey have legislation in place to allow for validation of 
non-formal learning and have certificated learners via this 
route, in the Middle East and North Africa and most of 
Eastern Europe, validation is not yet practised. In many 
cases, lack of qualifications standards against which to 
assess and certify is delaying more extensive use of this 
potentially powerful tool. 


Assessment 

Assessment Is the link between learner and qualifications 
content, vital in generating trust in the qualification offered 
to the individual. In the partner countries, assessment 
tends still to be traditional: school-based assessment 
by teachers and final written exams rather than practical 
tests. But there is change, driven by learning outcomes, so 
that assessment is more externalised, more modular over 
the course of a programme, and more practical. 


The examination centre in Montenegro is the external OA 
body for exams. It prepares, organises and conducts ex 
ams, and advises institutions. It trains examiners to assess 
NVQs. Georgia, has the national Assessment and Exami- 
nations Centre; in Azerbaijan the State Student Admission 
Commission is similar. 


The Federation of Employers in Ukraine is developing new 
VET qualifications derived from occupational standards. It 
is pushing for independent assessment bodies as in Mon- 
tenegro and Turkey. 


A different approach is applied in Kosovo: the NOA its too 
small to conduct assessments, SO assessors at providers 
must meet professional criteria and the NOA samples as- 
sessments for consistency. 


Certification 

Certification can either be of units or combinations of 
units. In partner countries, certification can be decentral- 
ised — issued by the provider — or centralised and issued 
by a national body. In VET, the range of certifying bodies is 
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greater, including such as Montenegro's examination cen- 
tre, or Turkey's VOC test centres. 


NQOFs usually allow these various categories of certifying 
institution, with the distinction in practice mainly in the type 
of qualification issued. QA should be aligned to account for 
these different types and certifying bodies, implying broad 
principles and criteria rather than more detailed control by 
the education ministry. 


In some countries, there is a clear distinction between an 
assessing institution and the one issuing the certificate. 
Trust in qualifications depends on the reputation of the is- 
suing body, arguably more than the institution that actually 
assesses the candidate. 


In Turkey, under the authority of the VOA, all qualifications 
must meet its criteria to enter the register. However, they 
are issued by the authorised certification body or VOC test 
centre that has conducted the assessment; there may be 
eight different authorised certification bodies for the same 
NVQ. To ensure that these qualifications are still recognis- 
able as national qualifications in the same field, all certif- 
icates have the same format, the logo of the authorised 
certification body (VOC test centre) and the VOA logo. 


2.3. Findings 


Countries are not blocked in implementing NOFs be- 
cause they do not understand the technical issues. On 
the contrary, the NOF concept and principles are well 
understood. Instead, progress is slowed by a failure to 
get organised and build or modify the necessary infra- 
structure. We have identified its four shared fundamental 
components as legislation, stakeholders, institutions and 
quality assurance. 


In the EU neighbourhood countries, legislation is usually 
necessary to effect qualification system reform, but the 
countries should first develop a broad reform strategy, 
identifying the NOF'’s role as catalyst for change, and then 
map existing legislation to target the proposed new legis- 
lation better. Primary legislation has too often been overly 
detailed. Where countries such as Kosovo have separated 
out principles — enacted via primary laws and operational 
arrangements, implemented through secondary regulation 
— the results have been better. Legislation should be com- 
prehensive of all key NOF functions. 
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Countries face a double challenge in engaging stake- 
holders in the qualification system: the private sector 
is relatively weak and unorganised, and lacks capacity 
to engage in training issues. Countries progress faster 
where they compensate for these deficits by setting up 
sectoral bodies or looking to chambers of commerce 
with their broad membership. Stakeholder bodies need 
to be institutionalised and given formal roles in educa- 
tion and training systems. Countries such as Turkey, 
where employers contribute to qualifications design, 
evaluation and quality assurance, see higher levels of 
trust in their VET qualifications. 


Institutional arrangements — allocation of functions and 
roles to official or formalised bodies — vary but we found 
similar challenges: building staff capacity to handle reform 
of the qualification system and managing strained budg- 
ets. Coordination is usually weak among the various insti- 
tutions. Most countries have never mapped functions and 
roles, so a first step is a thorough review of these, plus ap- 
praisal of capacity needs. Institutions should professional- 
ise and specialise. Countries benefit where they designate 
a lead body or, better still, create a specialised body: this 
has worked in Georgia, Kosovo, Montenegro and Turkey, 
which all concentrate functions and roles and expertise in 
a clear leading institution. 


The ultimate outputs of a qualification system are trust- 
ed qualifications. The infrastructure described above only 
has value if it results in that intangible, trust. Quality as- 
surance is found in the partner countries but is patchy, 
rarely Is It integrated into a national OA strategy. As with 
stakeholder contribution and _ institutional readiness, 
capacity and expertise are often insufficient. Countries 
should adopt national OA strategies, align assessment 
to standards and — as indicated also in our analysis of 
legislation, and institutions — engage stakeholders. Long- 
term, engaging stakeholders in certification processes 
will contribute to a quality culture. 


Making NOFs work means producing qualifications that 
have value for individuals in their career development. 
These modern qualifications need modern systems in 
place, in which an NOF can perform useful functions. So 
the 29 countries should concentrate on agreeing strate- 
gies, proportionate legislation, building sustainable institu- 
tions, coordinating, finding the right people, and dialogue 
and communication. 
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Summary 


As qualifications frameworks (QF) are increasingly intro- 
duced and implemented around the globe, consideration 
of their added value and impact on policies and practices 
is growing. Referring to European developments, but also 
taking into account the debates outside Europe, this chap- 
ter reflects on experiences and challenges in this area. 
Based on Cedefop’s national qualifications framework 
(NQF) monitoring since 2009 and supported by a range of 
comparative studies, the chapter outlines not only several 
areas where NOFs in Europe are making a difference but 
also areas where impact is less visible. Several methodo- 
logical and conceptual challenges in studying NOF impact 
are Identified. Finally, we highlight some initial indicators 
for evaluating the impact of NOFs. 


Keywords: national qualifications frameworks (NOFs); Eu- 
ropean qualifications framework (EQF); learning outcomes; 
evaluation and impact 


3.1. Introduction 


As qualifications frameworks (QF) are introduced all over 
the world, the question of their impact and added value is 
increasingly being asked. Do these frameworks, operating 
at sectoral, national and regional level, make a difference to 
policies and practices and to the learners and citizens they 
are supposed to benefit? This question is closely related 
to the challenge of measuring the impact of qualifications 
frameworks: what methods and reference point(s) can be 
used for measuring impact and how findings influenced by 
OFs interact and compete with numerous other initiatives 
in a broader policy context? 


The pioneering national qualifications frameworks estab- 
lished during the 1980s and 1990s have increasingly been 
confronted with these questions. National qualifications 
frameworks (NOFs) in Australia, New Zealand, the UK and 
South Africa have all been challenged over their relevance 
to and impact on policies, practices and learners. Similar 


questions are being asked of the new qualifications frame- 
works set up during the past decade. These new frame- 
works, vastly outnumbering the ‘early starters’, now have 
to prove their added value. 


The chapter first provides an overview of national qualifica- 
tions framework developments in Europe. The subsequent 
sections examine two preconditions of key importance 
for judging the impact of NOFs in Europe: the institution- 
al robustness and end-user visibility. Several areas where 
NQOFs are making a difference are highlighted, as are areas 
where impact Is limited. This impact alludes to monitoring 
of European NOFs carried out by Cedefop since 2009 and 
is also Supported by various comparative research projects 
covering different aspects of framework development. The 
section on impact studies from different countries identi- 
fies several challenges involved in studying the impact of 
NOF The final sections highlight some initial indicators for 
evaluating the impact of NOFs. 


3.2. National qualifications framework 
developments in Europe 


Developments in national qualifications frameworks in 
Europe have been rapid. Before 2005, frameworks were 
in place in 3 European countries: Ireland, France and the 
UK. By 2017, and directly triggered by the 2008 adoption 
of the European qualifications framework (EQF), NOFs 
have been introduced by all 39 countries (') taking part in 
the EQF implementation (’). 


(') The EOF covers the 28 EU Member States as well as Albania, 
Bosnia-Herzegovina, FYROM, Iceland, Kosovo, Lichtenstein, Mon- 
tenegro, Norway, Serbia, Switzerland and Turkey. 

(2) By June 2017, the following 32 countries had formally linked (‘ref- 
erenced’) their national qualifications frameworks to the EOF: Aus- 
tria, Belgium (Flanders and Wallonia), Bulgaria, Cyprus, Croatia, the 
Czech Republic, Denmark, Estonia, the Former Yugoslav Republic 
of Macedonia, France, Germany, Greece, Hungary, Iceland, Ire- 
land, Italy, Latvia, Lichtenstein, Lithuania, Luxembourg, Kosovo, 
Malta, Montenegro, the Netherlands, Norway, Poland, Portugal, 
Slovenia, Sweden, Switzerland, Turkey and the United Kingdom 
(England, Scotland and Wales). Slovakia and Romania are currently 
discussing their referencing reports with the EOF advisory group. 
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Most European countries have completed the (initial) con- 
ceptual and technical development of their frameworks. A 
total of 33 out of 39 countries that form part of the EQF 
process formally adopted their NOFs, most recently in 
Austria, Poland, Slovenia and Finland. Of the EU countries, 
only Italy and Spain have yet to finalise developments and/ 
or adoption (7). The Former Yugoslav Republic of Macedo- 
nia (FYROM), Kosovo and Montenegro, candidate or po- 
tential candidates for EU membership, also participate in 
the EOF, having already been referenced to the European 
framework. Albania, Bosnia and Herzegovina and Serbia, 
which joined the EOF process in autumn 2015, have legal- 
ly adopted their NOFs in various ways, establishing social 
dialogue platforms, and having achieved some technical 
advance such as producing outcomes descriptors. A grow- 
ing number of countries has reached what we may term 
an early operational stage. This is illustrated by frameworks 
in Austria, Croatia, Cyprus, FYROM, Greece, Hungary, 
Iceland, Kosovo, Luxembourg, Montenegro, Poland and 
Turkey. While still working on putting in place implementa- 
tion structures and adopting relevant tools and measures, 
these frameworks are now gradually starting to make a 
difference at national level. Several European NOFs have 
reached a more mature operational stage: Belgium (fl), 
Denmark, Estonia, France, Germany, Ireland, Malta, the 
Netherlands, Norway, Portugal, Slovenia, Sweden, Swit- 
zerland the United Kingdom. Despite having put all major 
features of their framework into practice, the challenge is 
to strengthen involvement, acceptance and ownership by 
key stakeholders and visibility towards end-users. While 
different in focus and objectives, some key common char 
acteristics of European NOFs can be identified: 


(a) most frameworks in Europe have been designed as com- 
prehensive frameworks, addressing all levels and types 
of qualifications from formal education and training (vo- 
cational education and training (VET), higher education 
(HE), general education) and, in some cases, opening up 
to qualifications awarded outside formal education and 
training. These frameworks can mostly be described as 


The remaining countries are expected to follow in 2017-18, which 
means that the first stage of EOF referencing is nearly finished. 
The completion of the first stage of referencing will be followed 
by regular updates in the event of major changes by the countries, 
which shows that EOF implementation is a continuing and ongo- 
ing process. Estonia and Malta presented updates in 2015, starting 
this phase (for Malta, this was the fourth update since 2009). 

(°) Italy has established the national repertoire of education, training 
and professional qualifications that consists of different sections, 
including a OF for HE and national framework of regional qualifi- 
cations. Development of a comprehensive framework is work in 
progress. Spain is finalising its framework for lifelong learning. 


(b 


(c 
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‘loose’ in the sense that they have been designed to 
embrace a multiplicity of education and training institu- 
tions and provisions, reflecting a broad range of values, 
traditions and interests. Whether a framework is tight or 
loose depends on the stringency of conditions a qual 
ification must meet to be included (Tuck, 2007 p. 22). 
Loose frameworks introduce a set of comprehensive 
level descriptors to be applied across subsystems but al- 
low, at the same time, substantial ‘differentiation’ within 
and between sub-frameworks (+). Tight frameworks are 
normally regulatory and define uniform specifications for 
qualifications to be applied within and across sectors. 
The comprehensive frameworks dominating in Europe 
can be understood as ‘loose’ in the sense that they inte- 
grate sub-frameworks (and their specific legislation) but 
refrain from introducing uniform rules for the design and 
award of qualifications. This is illustrated by the Polish 
qualifications framework where generic, national de- 
scriptors are supplemented by more detailed and target- 
ed descriptors addressing general, vocational and higher 
education. In many countries, this institutional diversity 
and the need to address a wide range of interests and 
concerns has put stakeholder mobilisation and commit- 
ment at the forefront of developments; 

while technical and conceptual design is important, 
creating commitment and ownership of the process, 
stakeholder buy-in, consensus building and overcom- 
ing resistance to change have been identified as crit- 
ical conditions for effective NOF development and 
implementation in Europe. This contrasts with some 
‘first generation frameworks’. Examples of early ver 
sions of frameworks in South Africa or New Zealand 
illustrate how attempts to create tight and ‘one-fit- 
forall’ variants can generate a lot of resistance and 
risk weakening the framework role. Such experiences 
have led to general reassessment of the role of these 
particular frameworks, pointing to the need to protect 
diversity (Allais, 2011c, Strathdee, 2011); 

the main objective of the new European frameworks, 
reflecting their ‘loose’ character, is to improve transpar 
ency of the national qualifications systems; the aim to 
make It easier for citizens to assess and make better 
use of national level qualifications, notably by clarify- 
ing how qualifications from different institutions and 
subsystems can be combined and support individual 
learning careers. This turns NOFs into key instruments 


A sub-framework is a framework, which covers only one subsys- 
tem (e.g. HE, VET) and is part of an overarching comprehensive 
framework. 
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for lifelong learning. Many countries have stated, when 
developing NOFs, that their purpose is not to reform 
the qualifications systems, but to make them more ac- 
cessible and easier to understand. As European NOFs 
have developed and moved towards operational status, 
however, countries increasingly tend to see them as 
contributing to incremental reform, influencing their 
overall impact on policies and practices. European 
NOFs also have strong international comparability, ad- 
dressing the need to support mobility of people across 
national borders. This positions European NOF develop- 
ments within broader human and social development 
and lifelong learning rather than within the neo-liberal 
policies they are traditionally associated with (Allais, 
2011a, 2014; Lassnigg, 2012). 


3.3. Sustainability and visibility of European NOFs: 
preconditions for impact 


The impact of European NOFs has only been partly as- 
sessed, reflecting their short history. Developed during the 
past decade, most frameworks have only reached a stage 
where indirect and partial impact can be assessed. From a 
survey carried out by Cedefop in 2015/16, however, the in- 
stitutional robustness and end-user visibility of the frame- 
works can be judged. These two factors can be considered 
as preconditions for impact and so are of key importance. 


3.3.1. Sustainability and institutional ‘robustness’ 

Most European countries now consider NOFs to be in- 
tegral parts of their national qualification system. They 
acknowledge that a strong legal basis with clear policy 
objectives is essential to guarantee and clarify the future 
role of the frameworks. Active and committed involve- 
ment of stakeholders, within and outside the education 
and training system, is also seen as a precondition for, 
and guarantee of, sustainability. Few countries explic- 
itly express doubts as to the future role of their frame- 
works. While some emphasise the need to clarify and/ 
or strengthen the political mandate, others point out that 
changing national political priorities risks influencing fu- 
ture implementation. While most countries are confident 
that their frameworks will remain in place, some point out 
that the frameworks’ ultimate impact will depend on in- 
tegration into mainstream policy processes. Even frame- 
works with a long tradition, such as that in UK-Wales, still 
need to be further streamlined and linked to education, 
training and skills strategies (Welsh Government, 2014). 
Politically and institutionally isolated frameworks will be 
less able to meet expectations. 
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Around one-third of countries (°) see NOFs as reform 
tools and expect them to support the restructuring, 
strengthening and/or regulation of their national qual- 
ifications systems. This contrasts with the position of 
countries at the start of implementation when frame- 
works were seen as instruments for describing qual- 
ifications, not for changing them. The potential of the 
frameworks to support reform have become apparent 
during initial development and implementation stages, 
differing slightly between sub-frameworks (°). Some 
countries, as illustrated by Ireland, have strengthened 
the link between the national frameworks and quality 
assurance bodies. Most flag the need for financial and 
human resources to be stepped up when NOFs reach 
the fully operational stage. 


3.3.2. Visibility 

To be of value to individual citizens, frameworks need 
to become visible. Many countries now systematically 
indicate NOF and EOF levels on the qualifications doc- 
uments they award (certificates and diplomas and Eu- 
ropass certificate and diploma supplements). It is also 
important that national (and European) qualifications 
databases contain this information and ideally structure 
information on qualifications in line with their frame- 
work. This visibility lies at the heart of the frameworks 
as tools for increased qualifications transparency. Coun- 
tries are making progress in this area. Denmark and 
Lithuania were the first to include references to EQF/ 
NOF levels in their VET certificates in 2012. By February 
2017, 21 countries had introduced level references in 
national qualifications documents or databases: Czech 
Republic, Denmark, Estonia, France, Germany, Greece, 
Iceland, Ireland, Italy, Lithuania, Latvia, Luxembourg, 
Malta, Montenegro, Norway, the Netherlands, Portu- 
gal, Poland, Slovenia, Switzerland and the UK. Sever 
al countries have indicated their intention to do so in 
2017, including Belgium (Flanders and Wallonia), Bulgar 
ia, Hungary and Austria. The visibility of frameworks to 
stakeholders outside education and training, particularly 
to employers, is still limited. Countries such as Ireland, 
the Netherlands, Sweden and UK-Scotland have made 
explicit efforts to demonstrate the relevance of frame- 
works to companies and branches. In most countries 
this is still an area to be explored. 


(®) Examples are Belgium (fl), Bulgaria, Croatia, Estonia, FYROM, 
Greece, Hungary, Iceland, Malta, Montenegro, Portugal, Slovakia 
and Turkey. 

(°) Examples are the Croatia, Iceland, Netherlands, Poland and Slovakia. 
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3.4. Early impact of European national 
qualifications frameworks 


The monitoring of European NOFs carried out by Cedefop 
since 2009, which is also supported by a variety of com- 
parative studies (’), points to several areas where NOFs 
are starting to make a difference. 


3.4.1. Improved transparency of national qualification systems 
The introduction of national qualifications frameworks with 
explicit learning-outcomes-based levels have helped to 
make national education and qualification systems more 
readable and easier to understand within and across Eu- 
ropean countries (Cedefop, 2016; 2017). In a few cases 
where multiple qualifications frameworks have been op- 
erating in parallel and partly in competition, as in UK-Eng- 
land, the impact on transparency is less clear. Introducing 
a common learning-outcomes-based language for describ- 
ing qualifications across education and training subsys- 
tems, the national frameworks provide a comprehensive 
map of national qualifications and relationships between 
them. Although differing between countries, important 
progress has been achieved in the past seven years (8). In 
some countries the increased transparency supports fur 
ther systemic reforms, as exemplified by Estonia where 
a lack of initial qualifications at NOF level 5 was identified 
through development of an overarching framework. The 
main discussion centred on the fact that there were no 
initial education and training qualifications identified at this 
level. Steps have now been taken to fill this gap. Following 
consultation with stakeholders, a new VET Act came into 
force mid-2013, introducing qualifications at level 5 (both in 
initial and continuing VET). 


This illustrates that the introduction of learning-out- 
comes-based levels, and the resultant placing of qualifica- 
tions, makes It possible to identify gaps in existing qualifi- 
cations provision. Cedefop’s relevant study (2014) shows 
that EOF level 5 (and the relevant NOF levels) has been 
used as a platform in developing new qualifications. This 


(’) See Cedefop publicationsinthe further reading section. NOF/EOF rel- 
evant Cedefop studies since the initiation of the EOF process in 
2005 also include: Annual NOF monitoring reports (2009, 2010, 2011, 
2012 and 2014) analysing national developments, including country 
chapters (for 2012 and 2014): http:/Awww.cedefop.europa.eu/en/ 
events-and-projects/projects/national-qualifications-framework-nqf 
Briefing notes addressing specific aspects of NOFs (2011 to 2016): 
http://www.cedefop.europa.eu/en/events-and-projects/projects/ 
national-qualifications-framework-naf 

(8) See EQF referencing reports: https://ec.europa.eu/ploteus/s|/ 
documentation 
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is exemplified by Estonia, Malta and the UK. In many oth- 
er countries (such as Estonia, Portugal and Slovakia) NOF 
level descriptors are used to review the content and out- 
comes of qualifications. Comprehensive and integrated 
qualification registers increasingly underpin the NOFs and 
make information on qualifications accessible for students, 
employers and guidance staff (°). Portugal exemplifies how 
NOF level descriptors are used to support the review and 
renewal of qualifications. 


3.4.2. More consistent implementation of learning 

outcomes approaches 

European NOFs share a common conceptual basis with 
a focus on learning outcomes. Recent research on the 
shift to learning outcomes (Cedefop, 2016) shows that the 
outcomes principle has been broadly accepted among na- 
tional policy-makers and that national qualifications frame- 
works have contributed significantly to strengthening this 
dimension. While the approach was previously taken for 
ward in a fragmented way in separate institutions and sub- 
systems, the emergence of comprehensive frameworks 
has made it possible to implement learning outcomes in a 
more system-wide and — to some extent — more consist- 
ent way. In countries such as Belgium, Croatia, Greece, 
Iceland, Lithuania, Norway and Poland the introduction of 
frameworks has led to identification of areas where learn- 
ing outcomes have not been previously applied or where 
these have been used in an inconsistent way. 


The level descriptors of the frameworks are increas- 
ingly used as reference points for describing, writing 
(and levelling) qualification and assessment stand- 
ards, as well as curricula. This is an important use of 
frameworks as it can strengthen the consistency of 
programmes and allow qualifications to be delivered 
according to similar requirements. 


The frameworks emerging after 2005 differ from the first 
generation frameworks developed, for instance, in UK-Eng- 
land or South Africa. Comprehensive NOFs in Europe can 
be categorised as outcomes referenced ("°) (Raffe, 2011b, 


(°) See for example the German qualifications database: https:// 
www.dar.de/content/2316.php or the Slovenian qualification data- 
base: http:/Awww.nok.si/en/ 

Raffe (2011b) explores different types of NOFs and examines the 
role of learning outcomes within them. He elaborates on two 
contrasting types of NOFs: outcomes-led and outcomes — refer 
enced; he suggests that these are associated with different roles 
for learning outcomes in pursuing the objectives of NOFs. A com- 
munication framework is typically outcomes-referenced. Learning 
outcomes-based level descriptors provide common reference 


(10 
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Cedefop, 2015) where the learning-outcomes approach — 
considered essential for levelling and increasing transpar 
ency of national qualifications — is linked to national cur 
ricula or programmes and accredited providers, accepting 
that mode and volume of learning matters. However, many 
frameworks have elements of the outcomes-driven mod- 
el where learning outcomes are specified independently 
from curriculum and provider (Ratfe, 2011b). This is most 
visible in some sub-frameworks as is the case in occupa- 
tional/professional qualifications in Belgium-Flanders, Es- 
tonia, Slovenia or Slovakia. For instance, in the Slovenian 
sub-framework of national vocational qualifications (NVQs) 
only qualifications and assessment standards are regulat- 
ed at national level. There are no formally accredited pro- 
grammes leading to these qualifications. 


It follows that the objectives and impact of the NOF will 
differ across sub-frameworks while a comprehensive 
framework will increase consistency of use of learning 
outcomes across sub-frameworks. 


3.4.3. Linking qualifications frameworks and validation of non- 
formal and informal learning 

The 2016 update of the European inventory on validation (") 
confirms that countries ('2) now give high priority to linking 
frameworks and validation arrangements. NOFs, through 
their focus on learning outcomes, act as a reference point for 
identifying, documenting, assessing and recognising learn- 
ing acquired in non-formal and informal settings. The intro- 
duction of NOFs allows countries to move from fragmented 
use of validation to a more coordinated national approach. 
According to the European inventory, there is a link between 
validation arrangements and formal education qualifications 
in the NOF in at least one education sector in 28 countries; 
this offers a possibility to acquire a full qualification or parts 
of a qualification included in the NOF. Links with validation 
are more common in initial vocational education and train- 
ing (IVET) and continuing vocational education and training 
(CVET) and higher education (HE) than in general education 
or adult education (in many countries not yet included in 
the NOF). Validation of non-formally and informally acquired 


point for diverse qualifications from different sectors and institu- 
tion and help coordinate education and training provision and im- 
prove coherences and integration of the system. 

(") Cedefop; European Commission, ICF (2017). Update to the Euro- 
pean inventory on validation of non-formal and informal learning 
— 2016 update. Synthesis report. Luxembourg: Publications Office. 
http://www.cedefop.europa.eu/en/events-and-projects/projects/ 
validation-non-formal-and-informal-learning/european-inventory 

(4) EU-28, EEA EFTA countries, Switzerland and Turkey are included 
in the inventory. 
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competences and skills is possible in 17 countries included 
in the report. Around 75% of countries use the same stand- 
ards in formal education and validation in at least one sector. 


3.4.4. Greater stakeholder engagement and coordination 

The new generation of comprehensive NOFs has helped to 
bring together stakeholders from different subsystems in 
education who do not routinely cooperate (VET, HE, general 
education) and stakeholders from education and employ- 
ment. In some countries comprehensive NOF development 
has brought together stakeholders from different sectors 
for the first time. Evidence shows that this cooperation has 
increasingly been formalised and institutionalised (Cedefop, 
2017). This is important to support the coherent implemen- 
tation and maintenance of the NOF across sectors and 
institutions. Cross-sectoral bodies such as national qualifi- 
cation councils have been established in Croatia, Cyprus, 
Estonia, Montenegro and Poland. In Austria, the NOF ad- 
visory board of seven experts assists the NOF coordinating 
body in allocating qualifications to levels; the NOF steering 
group (NOR-Steuerungsgruppe), has 30 representatives of 
all the main stakeholders (all federal ministries, social part- 
ners, stakeholders from the different fields of education and 
Lander). A coordination point for the German qualifications 
framework was set up in a joint initiative of the Federal Gov- 
ernment and the Lander in 2013. It has six members, includ- 
ing representatives from the Federal Ministry of Education 
and Research and Federal Ministry of Economics and Tech- 
nology, the standing conference of the ministers for educa- 
tion and cultural affairs of the Lander, and the conference 
of ministers for economics of the Lander. Its main role Is to 
monitor the allocation of qualifications according to levels 
descriptors, to ensure consistency of the overall structure 
of the framework. The impact of strengthened cooperation 
on progression routes across subsystems still needs to be 
seen. Deeper insights into acceptance and ownership of 
stakeholders in relation to the NOF objectives and imple- 
mentation are needed in the coming years. 


3.4.5. NOFs have opened up to qualifications awarded outside 
formal education and training 

Most European NOFs cover qualifications from formal edu- 
cation and training (VET, HE, general education). These are 
qualifications regulated and awarded by national authorities. 
However, there is a growing trend among countries to open 
up their frameworks to include qualifications awarded in 
continuing and further education and training, often awarded 
outside the formal national qualification system. Countries 
including the Netherlands, Slovenia and Sweden have start- 
ed developing criteria and procedures for inclusion of qual 
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ifications awarded outside formal education and training or 
have already included an important number of those qualifi- 
cations, as in the Netherlands. The Swedish national qualifi- 
cations framework has, from the start (in 2009), been seen 
as a tool for opening up to qualifications awarded outside 
the public system, particularly in adult education and in the 
labour market. Linking this ‘non-formal’ sector to the NOF is 
seen as crucial for increasing overall qualifications transpar 
ency and relevance in Sweden. 


3.4.6. NOFs and higher vocational education and training 

In many countries vocationally oriented education and 
training at higher levels have been operating in ‘the shad- 
ows’ of universities. This lack of visibility partly reflects 
high esteem attributed to academic and research-oriented 
education, in some cases undermining the role of voca- 
tionally oriented and practice-based education and training 
(‘academic drift’). The learning-outcomes-based levels of 
the NOFs have played a role by making visible the exist- 
ence and importance of vocationally oriented education 
and training at levels 5 to 8 of the EOF There is now an 
intensive debate on the future of vocational education and 
training at EOF levels 5 to 8 ('’). The increased visibility 
of higher vocational education and training (HVET) can be 
illustrated by placing the German master craftsman quali- 
fication at level 6 of the German qualifications framework, 
firmly underlining that vocationally oriented education and 
training can take place at all levels ("*). The Swiss national 
qualifications framework is explicitly designed to support 
this principle, showing how vocational and professional 
qualifications operate from level 2 to level 8 of the frame- 
work. The rapid development of HVET policies in many 
countries can partly be seen as directly influenced by the 
outcomes-based perspective provided by the NOFs and 
their learning-outcomes-based levels. This example shows 
that NOFs can make vocational qualifications at these lev- 
els more visible and contribute to increased diversity of 
qualifications designed for different purposes. 


3.5. Areas where less impact can be observed 


3.5.1. Limited visibility and labour market use 
Several evaluation studies have pointed to the limited 
visibility and use of NOFs by labour market actors (Allais, 


(3) Several research and development projects are currently exploring 
this area. A good example is the Beehives project: https://www. 
eurashe.eu/projects/beehives/ 

(4) Supplemented by policies and projects: http://ankom.his.de/pdf_ar 
chiv/2009_03_06-Hochschulzugang-erful-qualifizierte-Bewerber.pdf 
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2017; NOAI, 2009). The most successful example of good 
framework visibility on the labour market is the French 
NOF (known as national register of vocational qualifica- 
tions, Repertoire national des certifications profession- 
nelles), where qualifications levels are linked to levels of 
occupation, work and pay (Allais, 2017). 


In less regulated labour markets, the visibility and use by 
employers is more challenging. For instance, the evalua- 
tion of the UK-Wales framework concluded that too few 
employers engaged in or were aware of the framework. 
While this reflected a general lack of visibility of the cred- 
it and qualifications framework for Wales (COFW), some 
stakeholders pointed to the fact that the UK (England 
and Northern Ireland) qualifications and credit framework 
(OCF) was the dominant framework in the UK and that 
some employers may have preferred to relate to this and 
not limit themselves to UK-Wales (Welsh Government, 
2014, p. 45). The impact study of the Irish QF has demon- 
strated that it has considerable potential to be used in 
recruitment, in developing career pathways, in planning 
work-based learning and training and in recognising trans- 
ferable skills (NOAI, 2009). However, its use by employ- 
ers is limited, reflecting low awareness and visibility. A 
recent study carried out in Germany on the potential use 
of the German qualifications framework (Bundesministe- 
rium fur Bildung and Forschung, 2017) identifies several 
areas where the German qualifications framework (DOR) 
can add value. The framework as stated can, for example, 
be used to support human resource development (recruit- 
ment and development of employees); this applies espe- 
cially to small and medium-sized enterprises with limited 
human resource capacity, but will require capacity build- 
ing and awareness raising. UK-Scotland stands out as 
an exception in this area, having developed and promot- 
ed a range of tools that support employers in using the 
Scottish credit and qualifications framework (SCQF) and 
guides to support recruitment and staff selection, identify 
and plan skills development for staff or gain recognition of 
in-house training programmes ("®). There are examples of 
employerled sub-frameworks of vocational/occupational 
qualifications, for instance in Estonia, Slovenia and Turkey, 
with good use and visibility on the labour market, includ- 
ing access to regulated occupations, certification of the 
skills acquired at the workplace, recruitment, workforce 
development, and guidance. 


(') http://scqf.org.uk/employers/what-are-the-benefits 
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3.5.2. Articulation between institutions and education and 
training subsystems 

Several countries see their NOFs as tools for strengthening 
the links between education and training subsystems. This 
is considered essential for reducing barriers to progression 
in education, training and learning and for strengthening 
overall permeability of education and training systems. The 
new generation of European NOFs overwhelmingly con- 
sists of comprehensive frameworks, addressing all types 
of qualifications at all levels of formal education and train- 
ing. This means that they — through their descriptors — must 
be relevant to diverse institutions pursuing a wide variety 
of tasks according to different traditions and cultures. Ac- 
cording to Young and Allais (2009; 2011), one of the funda- 
mental challenges comprehensive frameworks face is to 
take account of epistemological differences in knowledge 
and learning that exist in different parts of education. It is 
generally too early to say whether the NOFs are making a 
difference in this area: any future impact study needs to 
address this ‘bridging function’ of the frameworks and as- 
sess whether individual learners are becoming more able 
to move horizontally and vertically and combine education 
and training from different institutions and subsystems to 
benefit their lifelong learning careers. 


3.5.3. Institutional reform: work in progress 

NQOFs have contributed to institutional reform in a limit- 
ed number of cases. Ireland, Greece, Malta, Portugal and 
Romania exemplify this through their decisions to merge 
multiple qualification bodies into single entities covering all 
types and levels of qualifications. The synergies gained in 
bringing together functions under one roof can speed up 
implementation. Other countries have indicated future in- 
stitutional reforms in this area, either in the form of merg- 
ers of existing institutions or in the form of new bodies, 
as in Croatia. This shows that comprehensive NOFs, even 
in cases where their main role is perceived as promoting 
transparency, can trigger institutional reform. 


3.5.4. NOF support to recognition of qualifications across 
countries: at an early stage 

At the moment the effect of the qualifications frameworks 
on mobility of learners and workers is still uncertain (ICF 
GHK, 2013), as implementation of the EOF (and many 
NOFs) is at an early stage. However, evidence gathered by 
a study on the (potential) role of qualifications frameworks 
in supporting worker and learner mobility (European Com- 
mission and DEEWR, 2011) shows that there are great 
expectations of qualifications frameworks in this through 
better recognition of qualifications. NOFs give important 
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information about the level of qualification and its link to 
other qualifications, as well as what the holder of a qualifi- 
cation is expected to know, understand and be able to do. 
The information on learning outcomes, workload, type of a 
qualification and quality assurance are important elements 
in formal recognition of qualifications. The subsidiarity text 
to the Lisbon recognition convention (UNESCO and Coun- 
cil of Europe, 2013) underlines that frameworks should be 
used systematically as a source of information supporting 
recognition decisions. 


3.6. Lessons from impact studies 


The research literature identifies several challenges in- 
volved in studying NOF impacts. These challenges in- 
clude, for example, the complexity of the field (Pilcher et 
al, 2015; Lassnigg, 2012; Lester, 2011) and the problems 
involved in gathering reliable evidence on success and re- 
liability (Allais, 2011a, 2014; Ratfe, 2009b, 2011a, 2011b, 
2012, 2013; Coles et al, 2014). Several contributors have 
pointed to the particular problem of causality, stating that 
it is difficult to identify a direct link between the causes 
and effects of frameworks (Higgs and Keevy, 2007; Bol- 
ton and Reddy, 2011; Pilcher et al, 2015). Other literature 
suggests that NOFs are social constructs based on deeply 
rooted relationships and partly conflicting interests (Raffe, 
2012, 2013; Higgs and Keevy, 2007). This means that a 
study of impact needs to transcend restricted technical 
analysis and consider the social dimension of NOF imple- 
mentation and impact. 


The purposes and objectives of an NOF can change over 
time as exemplified by some first-generation frameworks. 
New Zealand and South Africa attempted to introduce uni- 
fied qualifications frameworks aiming to harmonise the 
way education and training systems were organised and 
managed. Based on a radical learning outcomes approach, 
these frameworks were transformational in their character 
(Allais, 2011c, Strathdee, 2011), signalling high ambition. 
Neither accepted nor supported by key stakeholder groups, 
the New Zealand and South Africa frameworks have sub- 
sequently gone through a series of revisions (Strathdee, 
2011; Allais, 2011c; Bolton and Reedy, 2015). 


It may be that this initial ambition, potentially exaggerating 
and ‘overselling’ the role of frameworks, has influenced 
the debate on NOF impact in a negative way. While most 
policy-makers and researchers agree that initial transfor 
mational objectives were unrealistic, this does not mean 
that other objectives cannot be addressed and achieved. 
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Understanding the impact of NOFs requires a realistic 
baseline, reflecting the way the framework is positioned 
in the national political and institutional landscape. The fol- 
lowing impact studies exemplify these challenges. 


The South African qualifications framework is the most re- 
searched (Allais, 2011a, b, c; Keevy and Blom, 2007; Taylor, 
2010; SAOA, 2003, 2005, 2014, 2017). Studying and assess- 
ing impact was seen as complex because the NOF compris- 
es multiple sectors, stakeholders and initiatives; direct lines of 
cause and effects could not be assumed (Bolton and Reddy, 
2015). It was also argued that studying impact can ‘/.../ not 
be neutral and would necessarily be affected by the power 
struggle enacted within broader NOF discourse in South Afri- 
ka' (Coles et al., 2014, p. 28). The NOF impact work in South 
Africa was carried out in three waves: in 2002-05, 2009-10, 
and from 2012 onwards, with several attempts to improve 
research design. The NOF Impact study 1 (SAQA, 2003) es- 
tablished 17 criteria/indicators grouped into four sets against 
which the impact of the NOF was measured. The specific 
objective of the NOF Impact study 2 (SAQA, 2005) was to 
establish a baseline against which future progress was meas- 
ured. Both studies gathered valuable information on imple- 
mentation, challenges, issues and development areas. A sec- 
ond (planned) cycle was cancelled and replaced by a review 
of the framework leading to 2008 SAQA Act and the estab- 
lishment of three sub-frameworks with three quality coun- 
cils (Bolton and Reddy, 2015). Subsequent research focused 
on three sub-frameworks (the general and further education 
and training, higher education, occupational qualifications) 
and the roles of four NOF agencies (the three quality coun- 
cils and the South African Qualifications Authority (SAQA)) 
in implementing a comprehensive framework (Taylor, 2010). 
According to this research, the South African NOF was seen 
as a device aiming to relate different parts of a segregated 
system to each other (Bolton and Reddy, 2015). ‘From that 
time it became clear that the different priorities and voices of 
these NOF organisations called for a theoretical framework 
that would enable the capturing of the differentiated voices; 
not to do so would run the risk of adopting an apolitical ‘tech- 
nicist’ approach’, as Bolton and Reddy (2015) expresses it. 


A new impact study, in 2014, took into account lessons 
learned in previous studies (SAQA 2003, 2005; Taylor, 
2010) as well as impact studies carried out in other coun- 
tries. The study conceptualised the NOF as an activity sys- 
tem related to the main objective: to link different parts of 
the education and training system. It unpacks the national 
policy, cultural and socioeconomic context in which the 
NOF operates. The main focus of the study was to review 


A 


the design and implementation of the NOF functions, to 
collect and analyse data on access and progression, and 
examine in depth the work of the three quality councils es- 
tablished under the 2008 SAOA Act (SAQA, 2014, p. 5) ("9). 
Only two broad ‘stretch indicators’ were selected for their 
potential to enable evaluation over time: ‘moves towards 
systemic integration’ and ‘beneficiary gains’ (”). 


The New Zealand national qualifications framework was 
designed with an ambitious aim of transforming education 
and training and adopting ‘a common system of measuring 
and recording learning’. Intended as a unifying and trans- 
formative framework that should serve multiple groups 
and stakeholders, it faced difficulties from the beginning 
(Strathdee, 2011). In the course of implementation many 
changes were made to accommodate the interests of dif- 
ferent stakeholders. By acknowledging the specific quali- 
ties and needs of the different subsystems, the NOF has 
(according to Strathdee, 2011) gained wider acceptance. 


In contrast, the Australian qualifications framework (AQF) 
was designed as a ‘loose’ framework with no legal base 
and no direct power in accrediting qualifications or quality 
assurance (Wheelahan, 2011b). In 2009-10, the Australian 
Qualifications Framework Council (AQFC) initiated a ma- 
jor review of the framework with extensive consultation 
across sectors. This was an ex-ante evaluation on how the 
strengthened AOF was likely to impact on, and be affect- 
ed by, education and training and labour market structures 
and processes. It focused on how qualifications are con- 
structed, how they are measured against each other and 
how they are used and valued (Buchanan et al, 2010). The 
study identified key issues/tensions and likely impact of 
a strengthened AOF in each of the subsystems (second- 
ary schools, VET, HE and industrial relations). The likely re- 
sponses of each sector and its institutions, and interaction 
with the regulatory frameworks were researched, as well 
as data necessary to make informed judgments. ‘The role 


(8) A broad theoretical framework was also developed to understand 
change. It comes from activity theory through elaboration of Enge- 
strom’s cultural historical activity theory (CHAT) (Engestrom 1987, 
2001, cited in Bolton and Reddy, 2015) aiming to map and study 
complex interactions between institutions and individuals and 
change processes. 

(’’) These two indicators were sufficiently general to enable SAQA 
and three quality councils to evaluate and interpret developments 
over time in their respective contexts. The ‘systemic integration’ 
indicator refers to the need to integrate the system that was deep- 
ly divided along demographic lines and different forms and status 
of knowledge. The second indicator on ‘beneficiary gains’ notes 
the importance of learning and developmental opportunities for 
learners of all ages and stages of life. 
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of the strengthened AOF is therefore two-fold — to pro- 
vide ‘credentials consistency’, as the levels structure, its 
descriptors and the qualification specifications strive to do; 
and to ‘manage differences’ between the disparate, sec- 
torally-based institutional interests, the aim being to create 
a better-connected tertiary education sector’ (Buchanan et 
al, 2010, p. 8). In 2011, a revised and strengthened AQF 
was published with explicit learning-outcomes-based lev- 
els, detailed qualification type descriptors and supporting 
policies on pathways and credit transfer. The knowledge 
dimension in qualifications became stronger and more ex 
plicit with important consequences for VET qualifications 
that must now include an educational purpose besides the 
vocational one (Wheelahan, 2011b; Coles et al, 2014). 


The Scottish qualifications framework (SOF) was evaluated 
in 2003 (Gallacher et al., 2005) to understand its expecta- 
tions and its impact on policies and practices, as well as 
study factors that led to particular responses from institu- 
tions. The impact study in 2013 (SCOF partnership, 2013) 
looked at levels of awareness, the perception and under 
standing of SCOF among learners, parents, teaching staff 
and management. This evaluation, based on a combination 
of focus groups, online questionnaires, face-to-face inter 
views and in-depth interviews gave valuable insight into 
the level of implementation of the framework. The findings 
demonstrate that the SCOF is widely recognised by learn- 
ers, parents and educational professionals in UK-Scotland. 
The evaluation is important also outside UK-Scotland as it 
provides research-based documentation on the impact of 
the framework at the level of end-users. 


In Ireland, an impact evaluation of the national framework 
of qualifications (NFQ) was carried out in 2009, research- 
ing the degree of implementation and the impact of the 
framework on education and training; it was a five-year in- 
vestigation, including background paper prepared by the 
National Qualifications Authority of Ireland (NOAI), reports 
from key stakeholders, engagement with stakeholders, 
case studies and a public consultation. Findings referred 
to different areas: implementation of the framework, im- 
pact on learners, learning outcomes and cultural change, 
and framework visibility and currency. A new policy impact 
assessment of the NFO is under way (2016-17): a survey 
is being carried out that seeks views on policy impact of 
the NFO on transparency of qualification, quality issues, 
lifelong learning and qualifications, employability impact, 
impact on teaching, learning and assessment practices, 
views on development and governance of the NFQ. It also 
seeks the views on future policy priorities for NFQ: com- 
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munication, quality assurance, regulation, progression, 
recognition and design functions. 


Evaluation of the Welsh framework with regards to the 
strengths, challenges and weakness of the framework 
implementation was carried out in 2013-14 (Welsh Gov- 
ernment, 2014). Several recommendations were made in 
2015, based on the findings of the evaluation: to support 
the COFW as a meta-framework that underpins future 
qualification strategies; to revise the aims and objectives 
so that it evolves into a ‘functional’ national qualifications 
framework which acts as a vehicle for describing the quali- 
fications system in UK-Wales; to simplify and raise the lev- 
els of understanding and profile of the COFW; and to move 
ownership of the quality-assured lifelong learning (QALL) 
pillar from government back to the sector with a view to 
making formal and non-formal learning less bureaucratic 
and more accessible. 


In the UK (England and Northern Ireland), evaluation of 
the qualification and credit framework (QCF) was carried 
out based on the background paper looking into practi- 
cal experiences in implementing the OCF between 2008 
and 2014. After extensive consultation throughout 2014 
and 2015, and following a review of the OCF (18), Ofqual 
— the qualifications regulator — withdrew the regulatory 
arrangements for the OCF and introduced the regulat- 
ed qualifications framework (ROF). The ROF is a sim- 
ple, descriptive framework which requires all regulated 
qualifications to have a level and size. It is supported by 
Ofqual’s General conditions of recognition ('%) and statu- 
tory guidance (*°). 


Few of the new qualifications frameworks established af- 
ter 2005/06 have been subject to systematic evaluation, 
apart from Cedefop’s regular annual monitoring since 
2009. There have been isolated academic studies carried 
out in a limited number of countries, as in Denmark and 


(8) Ofqual (2014). A consultation on withdrawing the regulatory ar 
rangements for the qualifications and credit framework. https:// 
www.gov.uk/government/uploads/system/uploads/attachment_ 
data/file/381547/2014-07-24-a-consultation-on-withdrawing-the- 
regulatory-arrangements-forthe-qualifications-and-credit-frame- 
work.pdf 

(9) Ofqual (2016). General conditions of recognition. (Ofqual/16/6023). 
https://www.gov.uk/government/uploads/system/uploads/ 
attachment_data/file/529394/general-conditions-of-recogni- 
tion-june-2016.pdf 

(9) Ofqual (2016). Guidance to the general conditions of recognition. 
(Ofqual/16/6068). — https://www.gov.uk/government/uploads/sys- 
tem/uploads/attachment_data/file/538339/guidance-to-the-gener 
al-conditions-of-recognition-July_2016.pdf 
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currently on the use of the German framework and the 
Dutch qualifications framework. In other countries priority 
has been given to initial development, pushing impact is- 
sues into the future. In the past two years, however, some 
countries (such as Latvia and Norway) have signalled in- 
terest in developing a more systematic approach to the 
measurement of impact of NOFs. 


3.7. Identifying initial indicators for evaluating 
impacts 


Several lessons may be drawn from the above experienc- 
es, potentially informing the way we assess and measure 
the impact of qualifications frameworks. 


First, it is important to acknowledge the particular charac- 
ter and role of national qualifications frameworks: as mul- 
tilevel, dynamic and evolving tools, their objectives might 
change over time. This is clearly demonstrated by some 
irst generation frameworks as it is by frameworks develop- 
ments in Some European countries where focus and ambi- 
tion have evolved. It is illustrated by shifting the functions 
rom mere communication and transparency to strength- 
ening functions such as quality assurance, progression or 
opening up to qualifications awarded outside the formal 
system. The relative complexity of comprehensive frame- 
works in Europe is illustrated by the fact that they are 
based on sub-frameworks referring to sectoral legislation, 
sectoral institutions and stakeholders. Sub-frameworks 
may differ in ambitions, objectives and degree of regula- 
tion. Comprehensive frameworks firmly aim at strengthen- 
ing lifelong learning policies and practices and seek to in- 
tegrate and coordinate education and training subsystems. 
They are tools that relate different parts of education and 
training and only rarely used to directly regulate the design 
and management of qualifications. 


Second, NOFs should not be understood as a single-pur 
pose focused intervention. Comprehensive frameworks, 
such as those now developing in Europe, address multiple 
stakeholders and objectives, sometimes pointing to differ 
ent directions. Assessing the impact of NOFs requires clar 
ification of the objectives of a comprehensive/overarching 
framework, and its sub-frameworks, and of how these in- 
teract in a specific political, institutional and social context. 
Clarifying these objectives, and how they complement or 
contradict each other, makes it possible to establish a base- 
line for assessing the effectiveness and efficiency of the 
framework. It is also important to note that NOF objectives 
may differ in different sub-frameworks, for example in voca- 
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tional education and training and in higher education. When 
discussing the impact of NOFs, the different types of NOF 
and the way they link to and integrate subsystems and pol- 
icy areas, matter (Raffe, 2011a, 2013; Pilcher et al., 2015). 
Focus on functions of NOFs over time can help capture 
changing policy priorities. The complexity following from 
these characteristics must be taken into account when as- 
sessing and measuring frameworks impact. The following 
points should be considered: 


(a) the impact of NOFs needs to be understood in relation 
to the social, political and institutional context in which 
they operate. Bolton and Reedy (2015) point to the draw- 
backs of impact evaluations that focus ‘on the observ- 
able’ and on ‘outcomes’ rather than on the factors and 
mechanisms that led to the outcomes. All evaluation and 
impact studies emphasise the importance of contextu- 
alisation of the NOF to provide a narrative within which 
the outcomes of an evaluation can be interpreted. With- 
out contextual links, there may be the danger of adopt- 
ing a technicist approach. It is also important to look at 
the mechanisms and success factors; 

the time dimension is an important factor influencing 
the focus of an evaluation or impact study from two per 
spectives: NOFs change over time, and some authors, 
such as Taylor (2010) emphasise that timing of meas- 
urement is important. He suggests that in the first two 
years of NOF implementation, measurement of the ar 
chitecture is possible; two to five years of implementa- 
tion are needed before effectiveness of implementation 
can be examined; and 5 to 10 years before the impact 
can be assessed; 

the focus and the perspective of the evaluation and im- 
pact studies matter. An impact study must make a clear 
decision on which aspect to address and from which 
perspective this is to be done: if the study focuses on 
strengths, weaknesses and challenges of implementa- 
tion and its effectiveness; if it focuses on the use of the 
framework by different stakeholders; or if the study fo- 
cuses on how NOF objectives are achieved and which 
change processes are used (?'); 

experiences from South Africa illustrate the challenges 
involved in assessing impacts of NOFs, particularly by 
addressing the complex causality involved. The approach 
developed in recent years emphasises the relational role 


(b 


(c 


(d 


(71) Raffe (2013) argues that different change processes might be used 
in different countries by which they try to achieve their objectives 
(introducing common language, stakeholder coordination, regula- 
tion, quality assurance). 
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of the framework, pointing to its ability to involve and 
commit stakeholders and to promote cooperation and 
communication. This qualitative approach, complement- 
ing and partly contrasting a ‘simplistic’ measurement of 
quantifiable facts, makes it possible to capture the com- 
plexity of the framework and its functions; 

measuring the impact of NOFs requires baseline(s) and 
a limited number of indicators that can monitor the im- 
plementation and (possible) evolution of an NOF over 
time, as the (2014) study on impact of South African 
qualifications framework shows. This study also called 
for differentiated baseline across sub-frameworks for fu- 
ture evaluations and for studying contextual enablers. It 
also emphasised the importance of triangulation of data. 


(e 


3.8. Key elements in a strategy for assessment and 
measuring of impacts 


Few of the new qualifications frameworks established in Eu- 
rope have been subject to systematic evaluation. However, in 
recent years, as frameworks are moving towards operation- 
al stage, some countries have signalled interest in a more 
systematic approach to measurement of impact. In 2015, the 
EOF advisory group — the body that coordinates and oversees 
EOF implementation — discussed principles related to the fu- 
ture assessment of impact of the EQF and NOFs. These dis- 
cussions, addressing impact at national and European levels, 
illustrate several challenges of a more systematic approach to 
NOF impact assessment and measurement. 


Members of the EOF advisory group agree that the im- 
pact of qualifications frameworks needs to be understood 
within a broader political and institutional context and 
could not be reduced to a question of simple ‘objective’ 
causality. An instrumental and technicist approach should 
be avoided. An evaluation needs to contribute to better 
understanding on how NOFs operate within complex 
and dynamic social, cultural, political, historical, techni- 
cal and economic contexts. The evaluation should go be- 
yond what works to include critical reflection about why, 
how, for whom and under what conditions NOFs work 
(O'Connor, 2015). This calls for a theoretical and concep- 
tual framework that would be able to capture contextual 
factors, enablers and hindrances. 


An assessment methodology must also be able to cap- 
ture the informed interpretations and opinions of the main 
stakeholders. It would need to give voice to different 
stakeholders across sub-frameworks. Only by doing this 
is it possible to collect reliable evidence on added value 
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and the place of the NOF in the overall context of policy 
interventions. It is emphasised that measuring impact re- 
quires a robust and agreed baseline and most probably dif- 
ferentiated baselines across sub frameworks. Such a base- 
line must refer to the key objectives agreed for the OF in 
question. The EOF advisory group further emphasised that 
the baseline should allow countries to clarify the stage of 
development of the comprehensive frameworks, as well 
as sub-frameworks, and so to provide a realistic starting 
point for the national assessments. The development and 
agreement on baseline should respect the fact that many 
countries are still at an early stage of implementation and 
provide them with a realistic starting point for the exercise. 
The focus of the evaluation would need to reflect this. 


To stimulate this debate, Cedefop drafted an initial base- 
line proposal. This proposal refers to the following two 
main elements: 


(a) the four strategic objectives defined by the EQF recom- 
mendation (transparency; mobility; lifelong learning; and 
modernising education and training); 

the key (innovative) elements of the qualifications frame- 
works (explicit level descriptors, the learning outcomes 
principle; the introduction of frameworks covering all 
types and levels of qualifications; and the involvement, 
acceptance and ownership of key stakeholders from ed- 
ucation as well as employment). 


(b 


Table 3-1 exemplifies how indicators can be developed using a 
combination of these two elements (””). It is important to keep 
in mind that a limited number of indicators would be applied 
that enable evaluation over time. Further, there is a need for 
differentiated indicators and a differentiated baseline across 
sub-frameworks (being part of comprehensive national frame- 
works in most countries) and a solid framework for capturing 
contextual enablers and voices of different stakeholders. 


While only indicative, the table points to a limited number 
of aspects which will need to be addressed when (and if) a 
more systematic assessment and ‘measurement’ strategy 
is to be put in place, based on a sound theoretical and ana- 
lytical framework. While the horizontal axis of the table (re- 
flecting the core objectives of the 2008 and 2017 revised 
EOF recommendation) is specific to the European context, 
the vertical axis refers to generic factors shared by many 
qualifications frameworks worldwide. The objectives out- 


(72) An additional table outlining parameters for European level im- 
pacts has also been developed, but is not included here. 
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Table 3-1. Draft baseline for assessing and measuring OF impact at national level 


KEY ELEMENTS 


OF OFs (*) 


Level descriptors 
and the extent to 
which they are 

known and used 


The learning 
outcomes principle 
and the extent 

to which it is 
implemented 


The introduction 
of qualifications 
frameworks 
covering all types 
and levels of 
qualifications 


The involvement, 
acceptance 

and ownership 
of relevant 
stakeholders 


STRATEGIC EOF OBJECTIVES 


INCREASE 
TRANSPARENCY 


EQF/NOF learning- 
outcomes-based 
levels are referred 
to in certificates and 
diplomas, in national 
education, training 
and employment 
databases. 


Learning outcomes 
approach has been 
implemented by 

all education and 
training sectors, 
for initial as well 

as continuing 
education and 
training. 


The relationship 
between all types 
and levels of 
qualifications are 
demonstrated by 
the qualifications 
framework. 


Stakeholders 

from education 

and training, 
employment, civil 
society and other 
relevant sectors are 
aware of, committed 
to and actively use 
the framework. 


a0)" (0) = 
Be) \\Ce) 
LEARNING 


Level descriptors 
are used as a 
reference point by 
all lifelong learning 
providers and 
stakeholders. 


Decisions on 
access, exemptions 
and recognition — 
including validation 
of non-formal and 
informal learning — 
refer to learning 
outcomes. 


The qualifications 
frameworks 
increase 
communication 

and cooperation 
between levels, 
institutions and 
education as well as 
economic sectors. 


The framework 

aids cooperation 
between 
stakeholders in 
lifelong learning, 

for example in the 
form of efficient and 
effective delivery of 
validation of prior 
learning. 


INCREASE 
MOBILITY 


Level descriptors 
used as a reference 
point by labour 
market as well 

as education and 
training stakeholders 
are comparable 
across institutions 
and countries. 


The extent to 
which the learning 
outcomes/ 
competence 
approach is 
presented ina 
format which is 
understood by 
society at large and 
by employers in 
particular. 


The qualifications 
framework covers 
all qualifications and 
certificates relevant 
to employers and 
job-seekers. 


The qualifications 
framework is known 
to, and seen as 
relevant and credible 
by, labour market 
stakeholders. 


MODERNISING 
EDUCATION AND 
TRAINING 


Learning-outcomes- 
based levels are 
used as a reference 
and calibration-point 
when developing 
new qualifications 
and when reviewing 
and renewing 
existing ones. 


The learning 
outcomes 
approach informs 
the articulation 
of standards, 
programmes, 
curricula, 
assessment and 
teaching. 


The qualifications 
framework forms 
an integrated part of 
national education 
and training policies. 


The qualifications 
framework has led 
to a broadening 

of participation 
and involvement 
in the governance 
of education 

and training 
(governance). 


(*) National context, priorities and objectives that have informed the implementation of the EOF will need to be taken into account. 


Source: Cedefop. 
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lined for the horizontal axis can be replaced by other objec- 
tives which are relevant in different national contexts. The 
approach has been discussed in the context of the EQF 
advisory group but not empirically tested. 


A broad range of qualitative and quantitative methods 
and tools would need to be applied. Qualifications frame- 
works, through their emphasis on learning outcomes can 
often provide the reference point and baseline required for 
measuring progress in different policy areas. Introducing 
learning-outcomes-based levels — capturing the intended 
and actual outcomes of the national education and training 
system — is crucial to any strategy aiming at qualitatively 
better outcomes and not just at increased enrolment and 
completion rates. An important discussion to come is how 
to develop baselines, supporting more systematic ‘meas- 
urement’ of outcomes of learning. 
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3.9. Conclusions 


Most qualifications frameworks worldwide have yet to 
reach maturity and full operational status. It is not a given 
that all these frameworks will reach a stage where they 
add value to education and training policies and practices 
and directly benefit learners and citizens. For this to hap- 
pen it is necessary systematically to monitor and ‘meas- 
ure’ their impact and context enablers. This chapter, most- 
ly referring to developments in European NOFs during the 
past decade, has identified the need to develop and agree 
on clear assessment and ‘measurement’ baselines. Euro- 
pean experiences and debates are relevant, but will need 
to be developed, both conceptually and in terms of empir 
ical testing, in the coming years. 
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QUALITY LIFELONG LEARNING: QUALIFICATIONS FRAMEWORKS 
AND MECHANISMS FOR RECOGNISING ALL LEARNING 


Madhu Singh, UNESCO Institute for Lifelong Learning 


Summary 


This chapter looks at how countries use national qualifi- 
cations frameworks and mechanisms for the recognition, 
validation and accreditation of non-formal and informal 
learning outcomes to promote quality lifelong learning 
systems. It provides a comprehensive framework for un- 
derstanding the conditions for organising a quality lifelong 
learning system by analysing initial indicators at macro, 
meso and micro levels. Findings to date show that quality 
does not only depend on the existence of policies, frame- 
works and standards alone, but also on the relevance of 
delivery and quality of recognising, validating and accredit- 
ing (RVA) arrangements at local level and their usefulness 
to the end-user. This is a major challenge. Drawing on ev- 
idence from the global inventory of regional and national 
qualifications frameworks (Cedefop, ETF UNESCO and 
UIL, 2017, Vol. 2), we elaborate on a set of indicators high- 
lighted by countries in national qualifications frameworks 
(NQF) and RVA policy and practice. These indicators would 
have a bearing on the monitoring and evaluation of the role 
of qualifications and recognition processes in creating life- 
long learning systems that work in a quality manner. 


Keywords: quality; lifelong learning; national qualifications 
frameworks (NOFs); recognition, validation and accredita- 
tion (RVA); formal, non-formal and informal learning out- 
comes; sustainable development goals (SDGs) 


4.1. Introduction 


The overall purpose of this chapter is to provide an over 
view of how countries use national qualifications frame- 
works (NOFs) and mechanisms for recognising, validating 
and accrediting (RVA) non-formal and informal learning as 
tools to promote quality lifelong learning systems. The de- 
velopment of NOFs and moves towards learning outcomes 
approaches, reforms in governance, and involvement of 
social partners and other stakeholders, has brought in- 
creasing demand to recognise learning outcomes from all 
learning settings. 


The fourfold aim of this chapter starts with analysing a 
set of initial indicators of quality lifelong learning, with- 
in policy and practice of countries’ NOFs and recognition 
systems, that aim to strengthen lifelong learning. A sec- 
ond aim is to analyse quality lifelong learning within a sys- 
tems approach. Assuring the quality of lifelong learning 
involves continuously organising and improving lifelong 
learning quality at all levels - macro, meso and micro — in- 
cluding the quality components of structures, processes 
and outcomes/results. The chapter emphasises the im- 
portance of defining indicators of quality lifelong learning 
at the micro level, the level of the end-user of the lifelong 
learning system, where the products of qualification and 
recognition processes are developed. It is also the lev- 
el at which RVA practices help individuals to maximise 
opportunities to demonstrate relevant knowledge, skills 
and competences fully and accurately. A third aim is to 
highlight the need for countries to be aware of contexts 
in which a lifelong learning system operates, such as the 
nature of skill formation systems and the way the educa- 
tion and training systems are organised. This contextu- 
al awareness is important as it determines policy needs 
and actions for quality lifelong learning and how differ 
ent stakeholder groups use qualification and recognition 
processes address these needs. Fourth is examining the 
extent to which NOFs and recognition mechanisms can 
promote a holistic lifelong learning system, reflecting 
the full range of dimensions and processes: harmonising 
social and economic objectives, as well as dealing with 
quality elements around reliability, validity and standardi- 
sation as well as flexibility and individualisation. 


The chapter starts by highlighting the importance given 
to lifelong learning and quality education in the new 2030 
global agenda of sustainable development. It then con- 
ceptualises the notion of ‘quality’ at the macro, meso and 
micro levels, and breaking down quality components into 
structure, processes and results. Finally, using evidence 
from the global inventory of regional and national qualifica- 
tions frameworks (Cedefop, ETF UNESCO and UIL, 2017 
Vol. 2), it elaborates on a set of indicators that countries 
have highlighted in NOF and RVA policy and practice. 
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These indicators would have a bearing on the monitoring 
and evaluation of the role of qualifications and recognition 
processes in creating lifelong learning systems that work 
in a quality manner. 


4.2. Lifelong learning and the 2030 sustainable 
development goals (SDGs) 


Lifelong learning is at the top of the 2030 global education 
agenda, with the sustainable development goal 4 (SDG 
4) demanding inclusive and equitable quality education 
and the promotion of lifelong learning opportunities for 
all by 2030. The agenda conceives lifelong learning as a 
process, one that begins at birth and continues through 
all stages of life (Figure 4-1). ‘This approach to education 
incorporates multiple flexible learning pathways, entry and 
re-entry points at all stages, ages, and strengthened links 
between formal and non-formal learning structures, includ- 
ing accreditation of knowledge, skills and competencies 
acquired through non-formal and informal education. (UN- 
ESCO, 2016, p. 17). What is also new to the 2030 global 
education agenda is the concern for equitable access to 
post-basic education and training for the young and adults 
(Target 4.3) compared to the dominant emphasis on basic 
education within the Education for all (EFA) goals. 


The 2030 global education agenda also focuses on ‘learn- 
ing outcomes’ and ‘skills’ that are relevant for the young 
and adults both in the world of work (Target 4.4), and in the 
context of global citizenship in a plural and interconnected 
world (Target 4.7) (UNESCO, 2016). 


‘Education of good quality cultivates the flexible skills and 
competences that prepare learners for diverse challenges. 
The focus of quality ensures that foundation skills — litera- 
cy and numeracy — foster additional higher order thinking, 
creativity, problem solving, and social and emotional skills’ 
(Ibid, p. 17). 


Lifelong learning comprises all learning activities undertak- 
en throughout life with the aim of improving knowledge, 
skills and competences, within personal, civic, social and 
employment-related perspectives. 


Diversity in education types is another important quality 
issue in lifelong learning. UNESCO's global education mon- 
itoring (GEM) report highlights that ‘Local context and di- 
versity shape both challenges and solutions. Rather than 
pushing individuals into a one-size-fits-all programme, a 
lifelong learning approach incorporates diversity into an 
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inclusive, equitable system. Education for sustainability 
reaches out to serve marginalised communities by using 
all types of education matching learning to context’. (UNE- 
SCO, 2016, p. 17). 


What also emerges from the new Global agenda of sus- 
tainable development is the heightened awareness of the 
new contexts in which a quality lifelong learning system 
needs to function. These contexts relate not only to SDG 
4 on education, but also include targets from the other 
16 SDGs, such as decent work, labour productivity, skills 
development, health and well-being, and climate change 
mitigation. Other international trends such as migration, in- 
equality, information communication technologies (ICTs), 
and ageing societies are key factors, with great potential 
to influence the creation of a quality lifelong learning sys- 
tem including recognising, validating and accrediting (RVA) 
of non-formal and informal learning. For example, assess- 
ment, validation and qualification processes would need to 
adapt to circumstances of adults and young people from 
migrant and refugee communities by providing appropriate 
assessment methods that are more flexible and support- 
ive of lifelong learning. Similarly, ICTs, which have opened 
up lifelong learning opportunities to many new users, both 
inside and outside the formal system, will have profound 
implications on quality assurance otf lifelong learning. Inter 
national trade has meant that several international stand- 
ards have started to influence local non-formal and adult 
and lifelong learning provision. Skills development and 
skills recognition in countries where over 80 per cent of 
the workforce acquires its vocational competences out- 
side the formal system is another issue with quality assur 
ance challenges. 


The more qualifications and recognition processes be- 
come accepted and visible outside formal educational con- 
texts, the more issues about quality and accountability are 
going to be raised. A quality lifelong learning system will 
increasingly depend on how qualifications frameworks and 
recognition mechanisms can be counted as legitimate and 
trustworthy tools to recognise skills, knowledge and com- 
petences, regardless of the origin of that learning. 


4.3. Macro, meso and micro levels of quality 
lifelong learning 


Quality assurance, with appropriate indicators, is important 
in providing transparency, information, trust and shared un- 
derstanding among countries of what constitutes a lifelong 
learning system of quality. The European quality assurance 
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Figure 4-1. Lifelong learning opportunities for all 


Self-directed, family-directed, socially directed learning, 
workplace, family, local community, daily life 


Informal: Family- 
leads tono _ based child 
award care 


Source: UNESCO (2016), p. 16. 


for vocational education and training (EQAVET) supports 
coherence and implementation through the use of com- 
mon indicators ('). By establishing a shared understanding 
among Member States of what constitutes quality, EQA- 
VET increases transparency, consistency, portability and 
recognition of qualifications and competences of learners 
across European countries. It creates a common reference 
tool that enhances exchange and trust, and mobility of 
workers and learners. 


Evidence from 34 examples of NOFs (Cedefop, ETF UNE- 
SCO and UIL, 2017 Vol. 2) from the developing world and 
emerging economies in Asia, Africa and the Latin Amer 
ica, as well as the developed countries of New Zealand 
and Australia, has shown that countries have developed 
a comprehensive list of policy objectives for NOFs, which 
serve as important indicators of a quality lifelong learning 
system. Countries tend to place the goal of lifelong learn- 


(") http:/Awww.eqgavet.eu/qc/tns/monitoring-yoursystem/evaluation/ 
EQAVET_indicators.aspx 


Incidental learning, reading newspapers, 
listening to radio, visiting museums 


ing as an overarching policy goal, to be achieved via the 
following indicators. 


From a systemic point of view, these indicators can be 
embedded in judgments about quality at the macro, meso 
and micro levels. A key consideration at the macro level is 
the extent to which political decisions taken together with 
stakeholders from all sectors can ensure the legal basis of 
RVA initiatives, as well as well as financial involvement and 
governance. The influence of the State and potential for 
activity from companies and the private sectors and other 
stakeholders is of crucial importance. RVA linked to qualifi- 
cations frameworks must be discussed explicitly with deci- 
sion-makers and with those involved with local stakeholder 
groups. The nature of the constellation of stakeholders will 
have an important influence on the creation of a holistic 
lifelong learning system at the national level. 


Another important quality dimension at macro level is 
linkages between qualifications frameworks and recog- 
nition processes and reforms in existing education and 
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training systems. How can instruments such as quali- 
fications frameworks and recognition mechanisms be 
effective in developing flexible progression pathways 
between, often, separated general and vocational edu- 
cation tracks, as well as supporting RVA as an alternative 
route to qualifications or credits within lifelong quality 
learning opportunities? How can qualifications and rec- 
ognition mechanism be effective in creating a quality life- 
long learning system by improving the image, status and 
quality of certain non-formal learning education and train- 
ing tracks. A case in point is the low image of vocational 
education and training, or that of adult education in many 
developing countries. 


At the meso level, a key consideration in the creation 
of a quality lifelong long learning system is standardisa- 
tion and quality assurance of qualification and recogni- 
tion processes. Shavit and Muller (2000, p. 443) define 
standardisation as. ‘(...) the degree to which the quality 
of education needs the same standards nationwide. Var 
iables such as teacher training, school budgets, curricu- 
la, and the uniformity of school leaving examinations are 
relevant in measuring standardisation’. Standardisation is 
a useful term to understand the structures, processes 


VOLUME | — THEMATIC CHAPTERS 


and outputs underpinning the organisation and quality 
of qualifications and recognition processes. On the in- 
put side, focus should be on RVA in relation to reference 
points such as curriculum, qualifications and occupation- 
al standards. Input also relates to the level of expertise 
of RVA personnel. Processes will refer to the role of reg- 
ulatory agencies, interinstitutional relationships and mul- 
ti-stakeholder partnerships. Agencies and partnerships 
are important for ensuring quality processes in the devel- 
opment of standards and maintaining tools and method- 
ologies. Certification and the accompanying entitlements 
on the output side and may explain whether RVA forms 
part of exit-based or entry-based systems. For example, 
entry-based systems are those where follow-up training 
institutions devalue certification. 


The pedagogical perspective comes into the discussion at 
the micro level. Here the focus is on the relevance of the de- 
livery of recognition processes in education, working life and 
civil society. NOF standards and recognition mechanisms do 
not themselves promote quality lifelong learning: this needs 
to occur through closer action at the workplace, individual 
and provider level, and through the creation of partnerships 
and articulations between institutions and agencies. 


Table 4-1. Macro, meso and micro level indicators for creating a quality lifelong learning system: role of 


qualifications frameworks and recognition mechanisms 


INDICATORS 


Meso level Governance of RVA 


Use of learning outcomes 


Linking NOF and RVA: utilisation of agreed standards in RVA 


Establishing and maintaining learning pathways 


(*) RVA: recognising, validating and accrediting; NOF: national qualifications framework. 


Source: Author. 
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Many approaches are available. On the one hand, the 
learning content or the standards used to compare the 
individual's evidence of prior learning need to be analysed 
in relation to learning outcomes. It needs to be ensured 
that learning outcomes are defined holistically, and not 
based on a fragmentary and non-integrative understand- 
ing. This means, for example, that in addition to technical 
skills, it is necessary to consider the situational orienta- 
tion and context. On the other hand, It is also important to 
consider whether the personnel involved in RVA are able 
to undertake comprehensive personal career planning 
processes for the individuals. The methods used and the 
kind of arrangements made are all important, as is asking 
if the employment system is included at the micro level 
with the necessary support services. Also of importance 
is the extent to which institutions and organisations (pub- 
lic or private, workplaces, industry, NGOs and communi- 
ty-based organisations, TVET and educational institutions) 
employers and employees associations, have a stake 
and interest in RVA processes and are able to ensure 
real benefits in terms of employability, lifelong learning 
and personal development. At the micro level questions 
of ownership and control as well as usefulness must be 
clarified (Bjornavold, 2001, p. 20). The participation of 
stakeholders and the role of information as highlighted 
by Eriksen (1995) are also important micro-level issues. 
The organisation of RVA, therefore, cannot be limited to 
questions of methodology. It is important for enterprises 
and institutions to trust and accept the results of RVA of 
non-formal and informal learning. 


The qualification and recognition processes must pay at- 
tention to quality processes at all levels. 


4.4. Different contexts as starting points in a 
quality lifelong learning system 


4.4.1. State and private involvement in framing RVA policy actions 
Several factors in country contexts influence what could 
be seen as a quality and holistic lifelong learning system. 
For example, the influence of the State and potential for 
activity from companies and the private sectors and other 
stakeholders, and the way the education and training sys- 
tem is organised, are of crucial important. Many countries 
have acknowledged the need to widen participation in edu- 
cation and training and promote lifelong learning opportuni- 
ties through RVA routes. However, this largely depends on 
the nature of those involved in skills development and skills 
recognition in a country. In Denmark, Finland, the Nether 
lands and Norway, while the main stakeholders in skills for 
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mation and skills recognition are the national authorities, 
social partners encourage skills recognition in companies 
and organisations. The Scandinavian countries started pub- 
lic policy with a legal framework: laws stipulate functions 
and criteria for RVA and also allocate tasks to specified in- 
stitutions, bodies and authorities. The involvement of social 
partners, including professional associations, is a key fea- 
ture of RVA legislation; this targets specific groups, such as 
adults lacking secondary education adults, who may bene- 
fit from participating in a process of recognition of non-for 
mal and informal learning. In Norway, principles anchored 
in legislation are reflected in the successive introduction of 
various elements which together comprise a national life- 
long learning policy package (Christensen, 2015). However, 
studies (Ore and Hovdhaugen, 2014) have shown that im- 
plementation is quite different from policy and legislation. 


In State-dominated systems such as Bosnia and Herzego- 
vina and Romania, there is little responsibility to date taken 
by industry. However, it is acknowledged by all stakehold- 
ers that there is need for more communication and co- 
operation among government entities/district/cantons, the 
education sector and the labour market. One of the main 
challenges in Romania is to link structures and stakehold- 
ers from vocational education and training (VET), higher 
education and the labour market in a more comprehensive 
qualifications framework and recognition system, creating 
opportunities for lifelong learning. 


In the USA, skills recognition and lifelong learning takes a lib- 
eral approach with a low level of State and company influence 
and high level of individual direction (Busemeyer and Tram- 
pusch, 2012, pp. 12-149). A significant level of RVA activity is 
undertaken despite lack of relevant government policies or 
legislation. The widespread model of skills formation and skills 
recognition in the workplace is given priority (Barabasch and 
Rauner 2012). The Workforce Investment Act (WIA) of 1998 
(Public Law 105-220) was instrumental in establishing a fund 
for adult basic education (ABE) services, which encourages 
the development of RVA pathways for low-skilled adults to in- 
crease their educational attainment and obtain higher skilled 
jobs. The fund targets at-risk youth, undereducated and/or 
unemployed/underemployed adults, youth and adults with 
disabilities, and English language learners (ELL). The govern- 
ing structure of higher education is locally controlled within 
each individual US State, although financial support is deliv- 
ered through a combination of individual, local, State and fed- 
eral funding. The skills formation system in Canada is more 
or less the same as in USA (see Pilz, 2017), despite the fact 
that college programmes have a stronger skills development 
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component than in the USA. Like the USA, Canada does not 
have legal frameworks for RVA: certification bodies for regu- 
lated professions have developed RVA practices for their juris- 
dictions and the Canadian sector councils have sponsored a 
range of initiatives to promote RVA in the workplace. 


In the developing world and in emerging economies, the 
dominant context is one of low levels of State and compa- 
ny influence in skills development and skills recognition. 
India and Mexico are examples. In India, stratification is 
high particularly because of the strict separation between 
general and vocational training (Mehrotra et al., 2014); the 
latter has low status. Skill formation in the Indian system 
is dominated by informal structures and processes, with 
vocational education and training institutions, certifications 
and formal curricula playing only a minor part. As a result, 
learning processes tend to be directly linked to practice 
and the potential for recognising unrecognised skills in the 
informal sector Is high. In Mexico, the education and train- 
ing system is highly stratified, with general and academic 
education strictly separated from the vocational track. Vo- 
cational training is unorganised and follows a ‘learning by 
doing approach’, mostly on the basis of private motivation 
(Kis et al. 2009). Given the highly stratified education sys- 
tem, and small formal VET system, the main issue is the 
social and economic pressure for young people to enter 
the workforce without completing their formal education 
and this trend is set to continue. Through RVA, India and 
Mexico are aiming to promote the recognition of outcomes 
of non-formal and informal learning and encourage the de- 
velopment of small enterprises. In Mexico, the concep- 
tion and development of Agreement 286 of the Ministry 
of Education (issued on 30 October 2000) (and associated 
agreements) is designed to give both workers and learners 
access to formal education programmes at vocational or 
professional level. This is guaranteed through equivalency 
with credits in the formal education system. 


The establishment of RVA routes into education and train- 
ing depend greatly on the extent to which the systems are 
highly stratified. Germany has the tightest link between 
academic success in the school system and eligibility to 
enter higher education. This means that introducing RVA 
routes into the German system effectively calls this tight 
link into question (Ore and Hovdhaugen, 2014) so RVA 
routes mainly come from the vocational side of education. 
Legislation exists in a range of relevant legal acts and regu- 
lations allowing institutions and government departments 
to develop mechanisms and practical arrangements for 
RVA, depending on the diversity of purposes of RVA and 
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different interests at stake. Social partners play an impor 
tant role in RVA legislation in Germany. The inclusion in col- 
lective agreements of arrangements for the recognition of 
experience-based non-formal and informal learning is par 
ticularly conducive to the development of RVA. A legal ba- 
sis for recognition of employee skills and qualifications in 
collective agreements is provided by Article 9 Section 3 of 
the Basic Law, in which freedom of association is defined 
as a fundamental right, and the Collective Agreements Act, 
which asserts the principle of the autonomy of collective 
bargaining. Employers and employees are free to agree 
on working conditions in companies with no regulatory 
intervention by the State. In addition to defining pay and 
working hours, this includes arrangements for training and 
continuing education (Germany. Federal Ministry of Educa- 
tion and Science (BMBF) 2008, p. 50). 


Denmark, Finland and Sweden, when compared with Ger 
many, have relatively weaker links between school suc- 
cess and eligibility for higher education. This has made 
RVA routes to higher education more likely. 


An important quality dimension of lifelong learning at 
macro level is a consideration of how qualifications frame- 
works and recognition processes in Member States are 
linked to reforms in their existing education and training 
systems. In many countries, especially with education and 
training systems that are highly segmented and stratified, 
teaching and learning processes are theoretically oriented 
with low level of relevance to practice (Brockmann et al. 
2011). In France, one response is legislation which aims 
to give every individual the right to apply for RVA. There 
are several other French laws, such as the Law of Decen- 
tralisation accompanying the Modernisation Law of 2002, 
which have given stakeholders and providers particularly 
in continuing vocational education and training (CVET), the 
power to implement RVA. The Law on Lifelong Vocational 
Training and Social Dialogue enables employees to access 
training outside working hours. This is gives employers an 
important role in RVA. 


The dominant issue in skills recognition in Australia is the 
separation of general and vocational education and the 
low status of vocational training. To respond, the govern- 
ment established the Australian qualifications framework 
(AQF), whose key features have been standardisation and 
integration of trade qualifications centred on workplace 
competence with other VET qualifications and higher edu- 
cation qualifications. The development of flexible progres- 
sion pathways was also a key objective. This supports mo- 
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bility between higher education and VET sectors and the 
labour market by providing a basis for RVA (Cedefop, ETF, 
UNESCO and UIL, 2017 Vol. 2). Recognition of non-formal 
and informal learning plays an important part in these ef- 
forts, particularly because increasing certification serves 
the aim of ensuring better integration of vocational edu- 
cation and training into higher education and better col- 
laboration with key stakeholders to encourage improved 
transition to work arrangements. 


Closing the gap between education opportunities for dif- 
ferent groups in society is an important goal for Australia, 
New Zealand, and South Africa particularly those of indig- 
enous groups raising the skill potential of workers who 
can and want to work or are currently excluded from the 
labour force (New Zealand Ministry of Education, 2008). 
With the large influx migrants and refugees into Europe, 
the recognition of migrants’ prior learning and experiential 
learning is expected to become an important integration 
policy issue in the coming years, with implications for 
quality. There is also emphasis on utilising existing poten- 
tial skills in the economy (BMBF) 2008). In Austria, recog- 
nition of non-formal and informal learning is seen as aiding 
the integration of marginalised groups such as migrants, 
elderly persons or the unemployed by giving them a ‘sec- 
ond chance’ (Austrian Federal Ministry of Education Arts 
and Culture, 2011). 


4.5. Meso level: regulatory frameworks 
and quality assurance 


The notion of quality must be understood not only at the 
policy framework and legislative level, but also at the 
meso level. This covers interinstitutional arrangements 
that link academic, vocational and continuing education 
programmes/institutions, and provide potential for awards/ 
credits or generic awards across levels, subjects, and 
sectors. Partnerships across sectors between stakehold- 
ers from education, industry, and community adult learn- 
ing sectors are important. These aspects of quality at the 
meso level need attention. 


Regulatory agencies such as national qualifications author 
ities and accreditation bodies must be able to harmonise 
minimum standards for accreditation of qualifications ob- 
tained through all learning: formal, non-formal and informal 
learning. Registrations for education providers and training 
institutions, as well as systems for assessing learning and 
issuing qualifications and certifications, are important com- 
ponents of a quality lifelong learning system. 


ae 


4.5.1. Governance of RVA 

RVA governance is an important dimension of quality 
assurance of a lifelong learning system in which the in- 
volvement of stakeholders and clear accountability struc- 
tures are required. 


Successful organisation of RVA is dependent on the extent 
to which various partnerships drive the coordination. In- 
formation on national policies and practices indicates that 
partnerships with stakeholders differ significantly. We refer 
to three models of implementation and coordination that 
emerge from the country cases. The elaboration of exam- 
ples does not aim to be exhaustive but rather illustrative. 


The social partnership model 

In Austria, Germany and Switzerland, multiple social partners 
and stakeholders treat implementing RVA as a shared re- 
sponsibility, coordinating their work in accordance with laws, 
regulations and guidelines. This ensures legitimacy within 
decentralised education. In Austria (Schneeberger, Petano- 
vitsch and Schdgl, 2008) responsibilities for the regulation, 
provision, financing and support of learning activities are di- 
vided between national and provincial levels. Social partners 
play a role in designing the legal, economic and social frame- 
work conditions. Education institutions organise or provide 
preparatory courses for exams and design other procedures 
to validate prior learning, based on their respective quality as- 
surance. In Germany, there is neither a central institution nor 
a standardised institutional framework in place for validation, 
but a variety of approaches. Chambers of crafts, industry, 
commerce and agriculture regulate admission to the exter 
nal student examination. The German Rectors’ Conference 
has defined a framework for recognition for higher education 
but specific regulations and procedures are established by 
the respective university. The ProfilPASS system is managed 
by a national service centre which supports 55 local dialogue 
centres. Responsibility for continuing education falls across 
several areas. Continuing education in Germany has less reg- 
ulation at national level than other areas of education and, as 
aresult, features a high degree of pluralism and competition 
among providers. Voluntary participation in continuing educa- 
tion is one of the guiding principles (German Federal Ministry 
of Education and Science (BMBF), 2008). 


National institutes such as Skills Norway, the knowledge 
centres in Denmark and the Netherlands are established 
under their respective ministries of education, which in 
turn cooperate with trade unions, enterprises, national 
labour agencies, national education associations, organi- 
sations, universities and colleges, public and private ed- 
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ucators, and social partners. Skills Norway is the body 
designated by the Ministry of Education to work on RVA 
at the national level. It is responsible for developing valida- 
tion guidelines for enrolment in tertiary vocational educa- 
tion and exemption in higher education. In 2013, the Nor 
wegian Directorate for Education and Training developed 
national guidelines for RVA in lower and upper secondary 
education. Skills Norway cooperates with NGOs and so- 
cial partners to further adult learning in working life. In the 
Netherlands, the EVC (Erkenning van Verworven Compe- 
tenties (recognition of acquired competences)) Knowledge 
Centre works in cooperation with a network of RVA re- 
gional offices. These are as one-stop locations where in- 
dividuals can access multiple services appropriate to their 
specific needs. In 2006, stakeholders agreed to a quality 
framework for RVA that, while voluntary, promotes trans- 
parency and articulates minimum standards (Maes, 2008). 
Individuals working through the available RVA structures 
are granted a certificate of experience to submit to edu- 
cation institutions. The certificate has the status of an ad- 
visory document and the ‘autonomous institutions decide 
for themselves how to use the results of EVC procedures’ 
(European Commission et al., 2010). 


Stakeholders in the adult and community learning sector 

The high level of individualisation in skills formation and 
skills recognition in the USA can be seen in the fact that 
prior learning assessment (PLA) is not governed by leg- 
islation (Travers, 2011). PLA is conducted in many colleg- 
es and universities and its policies and practices play an 
important role in several higher education institutions that 
have been serving the adult learner population RVA. These 
institutions gave birth to work on how outcomes from 
adult non-formal and informal learning could be assessed 
at an individual level. Funding for PLA services is gener 
ally the responsibility of individual education institutions. 
Fees are normally charged to the individuals undertaking 


Table 4-2. Governance in RVA 


GOVERNANCE 


Social partnership model 


Stakeholders in the adult and 
community learning sector 


Canada, USA 


NOFs coordinating RVA 
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assessment. The quality of higher education remains a top 
priority. Some of the six regional accreditation commis- 
sions located across the country have issued policies and 
guidelines on PLA that allow for varying degrees of institu- 
tional flexibility. For example, the New England Association 
of Schools and Colleges (2005) restricts individualised PLA 
to undergraduate level but allows flexibility in programme 
structure. The policy and practice for accepting prior lean- 
ing assessment credits, established by individual institu- 
tions, largely reflects local faculty agreements (Travers, 
2011, p. 251) and are responsible for monitoring the quality 
of higher education through formal accreditation. PLA de- 
velopment has been supported by the American Council 
on Education (ACE), the national body responsible for co- 
ordinating higher education institutions across the country. 
The Council for Adult and Experiential Learning (CAEL) is 
a national, non-profit organisation that works with educa- 
tion institutions, employers, labour organisations and other 
stakeholders to promote creative, effective adult learning 
strategies. Networks and structures such as CAEL aim to 
bring greater coherence to RVA in higher education. 


In Canada, prior learning assessment and recognition 
(PLAR) is a highly decentralised process with the responsi- 
bility for assessment and validation distributed across the 
various provincial/territorial governments, education insti- 
tutions and professional bodies. Adult educators have been 
at the forefront of RVA. The community of Canadian adult 
educators became acquainted with the work of CAEL in 
the USA and began to promote RVA in Canadian post-sec- 
ondary education, as highlighted in an article by Joy Van 
Kleef (2011). Their reasons for promoting RVA lay in the 
perceived nature of adult education as community-based 
and encouraging the development of knowledge and skills 
within a framework of lifelong learning. Three groups of 
adult educators — institutional practitioners, communi- 
ty-based practitioners and academic researchers — have 


COUNTRIES 


Austria, Denmark, France, Finland, Germany, Netherlands, Norway, Switzerland 


Australia, India, Mauritius, Mexico, Namibia, New Zealand, Pakistan 


Philippines, South Africa 


Source: Author. 
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been the primary sources of PLAR research in Canada. 
The Council of Ministers of Education Canada (CMEC) has 
responsibility for the Canadian Information Centre for In- 
ternational Credentials, which provides information on for 
mal credentials assessment services, provincial/territorial 
education systems, post-secondary institutions, regulated 
and unregulated occupations and how to connect with pro- 
vincial/territorial regulatory bodies that have responsibility 
for issuing licences to practice in each jurisdiction. Stake- 
holder engagement at the national level includes CAPLAs 
yearly conference that attracts a wide range of RVA stake- 
holders from across Canada and abroad. 


NOFs coordinating RVA 

A key element in RVA quality assurance is the develop- 
ment of national qualifications frameworks (NOFs). NOFs 
are an important part of education and training reforms in 
developed, transitioning and developing countries (Singh 
and Deij, 2016). They seem to have reached saturation 
point, with developments deepening within some coun- 
tries and not much progress in others, but NOFs continue 
to support education and training, skills development, and 
lifelong learning. Many countries consider that the exist- 
ence of a qualifications framework may help to promote 
systems for recognising non-formal and informal learning 
outcomes, especially in the case of recognition proce- 
dures formalised to the extent of awarding qualifications. 
NOFs provide a central reference point for the recognition 
of non-formal and informal learning. 


Written record of qualifications available through recog- 
nition of non-formal and informal learning outcomes can 
confer a status and form of legitimacy by associating them 
more closely with qualifications obtained via formal chan- 
nels. Where recognition of non-formal and informal learn- 
ing can lead to the award of a qualification, these quali- 
fications should be included in the national qualifications 
directory; this is necessary for securing social recognition 
and legitimacy. Standards should be accepted by all stake- 
holders and in particular by the various ministries that 
award qualifications, such as the ministries of labour and 
ministries of education. 


In the majority of countries surveyed RVA is subsumed un- 
der acts passed under the establishment of their NOFs. In 
South Africa the recognition of prior learning takes place in 
the context of the South African qualifications framework 
(SAQF) that came into effect with the passing of the South 
African Qualifications Authority (SAQA) Act of 1995. The 
Australian qualifications framework (AQF) aids progression 
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of students through qualifications by giving credit for learn- 
ing outcomes already achieved. Credit outcomes may allow 
for entry into a qualification or provide credit towards the 
qualification. Credit given may reduce the time required for 
a student to achieve the qualification. The organisation issu- 
ing RVA determines the extent to which previous learning 
is equivalent to the learning outcomes of the components 
of the destination qualification; it takes into account the 
likelihood of the student successfully achieving the qualifi- 
cation outcomes and ensures that the integrity of the qual- 
ification is maintained. The AQF qualifications pathways 
policy establishes the principle that pathways are clear and 
transparent to students and can facilitate credit for entry 
into, and towards, AOF qualifications. The Mexican qualifi- 
cations framework (MQF) is a comprehensive framework 
developed by the General Directorate of Accreditation, |n- 
corporation and Revalidation (Direcciédn General de Acred- 
itacién, Incorporacién y Revalidacién, DGAIR), within the 
Ministry of Public Education (SEP). Stakeholders from all 
sectors (industry, education and civil society) have partici 
pated in the development of the MOF CONOCER has been 
especially active on issues related to the national system of 
competency standards (NSCS) and on equivalences with 
formal education degrees. In October 2012, the Ministry of 
Education announced the new Mexican Bank of Academic 
Credits (announcement published by DGAIR on the official 
Mexican Government Diary of 4 October 2012, Article 8), 
which allows certificates of competence from CONOCER 
and from other recognised private and public training/certi- 
fication centres to be accredited as part of formal education 
programmes at lower and upper secondary levels. 


Evidence from Botswana, Ghana, Mauritius, Seychelles, 
and South Africa reveals that qualifications authorities, as 
models of shared responsibility between stakeholders, are 
prime movers behind the recognition of formal, non-formal 
and informal learning. In these countries, for all qualifica- 
tions validated by the qualifications authority and placed 
on the NOFs, there is now a requirement that recognition, 
as an alternative route to gaining qualification, should be 
possible. In most cases, the qualifications authorities elicit/ 
engage the support of professional organisations and em- 
ployment agencies to take forward recognition of non-for 
mal and informal learning and in seeking financial support 
for recognition initiatives. 


National qualifications frameworks alone cannot promote 
recognition of the outcomes of formal, non-formal and in- 
formal learning. Evidence from the NOF examples shows 
that countries face challenges at the meso level in ensuring 
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that structures such as regulatory agencies, qualifications 
authorities and accreditation bodies are able to harmonise 
on minimum standards for accreditation of qualifications 
obtained through all learning: formal, non-formal and infor 
mal. Another challenge is equipping education providers 
and training institutions to initiate RVA. Recognition prac- 
tices linked to NOF need acceptance by all stakeholders. 


4.5.2. The use of learning outcomes 

Learning outcomes have had an important impact on how 
levels and qualification standards have been defined. Their 
use can promote clarity and support participation through 
emphasising the relevance of programmes. However, 
learning outcomes require attention in several respects. At 
policy level they need to be overarching. At qualifications 
standards level they need to deal with intended learning 
outcomes. For learning programmes they need to deal with 
specific learning outcomes related to inputs and have a 
more pedagogical purpose. Learning outcomes should not 
be formulated in narrow and restricted ways which could 
limit rather than broaden the expectation of learners. From 
a lifelong learning perspective, learning outcomes need 
to reflect all contexts from life-wide, life-deep and lifelong 
learning perspectives. Even in countries without national 
qualifications frameworks, such as the USA and Canada, 
there is a trend in some institutions to design degree pro- 
grammes around student learning outcomes, or compe- 
tences, rather than college credits. Evaluative frameworks 
are being developed in increasing numbers for compe- 
tence-based prior learning assessment programmes to 
equate their effectiveness to other programme evaluation 
processes within institutions of higher education. The 
standards defined by the Council for Adult and Experiential 
Learning (CAEL) for competence-based prior learning as- 
sessment (PLA) are being interrelated with quality criteria 
used in the evaluation of college academic programmes; 
the aim is developing overarching evaluative frameworks 
that also embed the effectiveness of PLA programmes. In 
Canada measures for assessing education quality (CAEL 
standards for PLA) are applied to the assessment of prior 
learning in competence-based education and assessment, 
for example in professional registration. 


European Union countries are gradually beginning to grant 
non-formal and informal learning contexts the same value 
as formal learning. However, many institutions awarding 
non-formal qualifications need further assistance in de- 
scribing their qualifications in terms of learning outcomes 
to comply with the NOF requirements. 
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4.5.3. Linking NOF and RVA: using agreed standards in RVA 

Using agreed standards is an important feature of alter 
native recognition routes or/and credit transfers leading 
to a qualification. Agreed standards put outcomes from 
non-formal and informal learning on an equal footing with 
those from formal learning: in existing formal curricula; 
learning-outcomes-based qualifications standards; and oc- 
cupational standards. 


One way of achieving a qualification is to standardise 
second-chance education in relation to a curriculum with 
equivalence to formal education. In Belgium, the Second- 
ary School for Adults (SSA) is part of the regular education 
system and organises courses appropriate for adults. 


A second way to achieve a qualification is by aligning 
non-formal and informal learning outcomes to reference 
standards in NOFs. In Ghana, alternative transition routes 
to qualifications are proving beneficial especially for the 
young and young adults who have little access to formal 
education but have acquired their skills, learning and work 
experience in non-formal and informal settings. 


A third way of achieving a qualification is in relation to oc- 
cupational standards in a qualification. Countries are in- 
creasingly using RVA mechanisms within industry sectors 
as a means of enhancing the economic capacity of the 
workforce. Mauritius is making concerted efforts through 
the Mauritius Qualifications Authority (MOA) to recognise 
experience and learning of workforce in industry sectors 
such as tourism, financial services, real estate, information 
and communication technology (ICT) and seafood. 


Reference standards, regardless of whether they are for 
mal curricula or learning-outcomes-based qualifications, or 
occupational standards, do not necessarily have to lead to 
a full qualification; they can also lead to a partial qualifica- 
tion such as credits, which individuals transfer and accu- 
mulate towards achieving a qualification. Hong Kong SAR 
China has provision for credit accumulation and transfer 
(CAT) in its assessment systems, and to see recognition 
of non-formal and informal learning as an assessment and 
credit process. The CAT system allows learners to accumu- 
late credits from diverse courses and all modes of learning 
— including class attendance, private study, online learning, 
practical learning, examination — and convert them into a 
recognised qualification. It provides the flexibility needed 
to suit individual learners’ circumstances and minimise du- 
plication of training. 
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In Indonesia, a credit transfer scheme is a part of bridging 
programmes, aiming to boost the performance of individ- 
uals who fail to meet the minimum requirements of the 
institution conferring their credits. In the Republic of Korea, 
the lifelong learning account (LLA) system, in place since 
2008, allows an individual's diverse learning experiences to 
be accumulated and managed within an online learning ac- 
count; learning results are recognised as education credits 
or qualifications so that each individual has opportunities to 
find a better job. The national education system comprises 
formal and non-formal learning, which has equivalences 
at all levels. In New Zealand, recognition of prior learning 
(RPL) is part of the credit system. Education providers (in- 
cluding employing organisations, industry and professional 
bodies and education organisations, and several institutes 
of technology and polytechnics) are required to have their 
own administrative and practical arrangements in place for 
RPL and credit recognition and transfer. 


RVA as an alternative route to a full or a partial qualification 
offers people who have learned on the job an important 
second chance pathway to retraining and upskilling op- 
portunities. Qualifications frameworks help to clarify the 
formal demands in qualifications, rendering them more 
understandable and transparent for the young to manage 
transitions. NOFs increase the sense of security that the 
outcomes of learning will at least be of quality and equal 
to a national standard. At the same time, adults and the 
young have the quality and relevant skills demanded by 
employers and other stakeholders (Cedefop, 2009). Lo- 
cally defined qualifications, in contrast, may only offer this 
security to those who have access to good education in- 
stitutions and receive a certificate issued by an accredited 
training provider. NOFs as communication tools are, there- 
fore, important and should inform transition and lifelong 
learning policies. 


4.5.4. Establishing and maintaining learning pathways 

Among the many broad goals of countries in establishing 
NOFs is supporting the development and maintenance of 
pathways for accessing qualifications, and assisting peo- 
ple to move (vertically and horizontally) easily between 
different education and training sectors, and between 
qualifications of the different subsystems. The concept of 
learning pathways is important in lifelong learning. A learn- 
ing pathway is a sum of learning sequences followed by an 
individual to acquire knowledge, skills and competences. 
It may combine formal and non-formal learning sequences 
through the validation of outcomes from non-formal and 
informal learning (Cedefop, 2014). 
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The global inventory of regional and national qualifications 
frameworks (Cedefop, ETF UNESCO and UIL, 2017, Vol. 2), 
illustrates distinctive learning pathway types in the NOFs 
of developed, developing countries and emerging econo- 
mies. Table 4-3 presents five pathway types. 


Countries with highly developed qualifications frame- 
works have explicit pathway strategies. Both New Zea- 
land (NZQF) and Australian qualifications frameworks 
(AQF) have an explicit ‘pathway strategy’. In the case 
of the New Zealand qualifications framework, each 
qualification outcome statement includes the following 
pathways. Graduate profiles identify the expected learn- 
ing outcomes of a qualification: these describe what a 
learner will know, understand and be able to do when 
they achieve the qualification. Educational pathways 
identify the qualifications a graduate can enrol in after 
completing the qualification in question. Where qualifi- 
cations are stand-alone and do not prepare graduates 
for further study, the outcome statement should make 
this clear. Employment pathways (or contribution to the 
community) identify the areas in which a graduate may 
be qualified to work, or the contribution they may make 
to their community having achieved the qualification in 
question (Keller, 2013). 


Similarly, the new documents of the strengthened AQF 
in 2011 stipulate that by 2015 all courses, whether in VET 
or higher education, must be compliant with the AQF and 
need to show how they provide preparation for ‘further 
learning’ as well as for work. To underline this requirement, 
the AOF document incorporated a strengthened ‘pathways 
policy’ (AQF Council, 2013). 


Continuing and post-secondary and employment path- 
ways to quality-assured qualifications are found in coun- 
tries and territories such as Hong Kong SAR, China. 
Here the qualifications framework is a tool to articulate 
the academic and vocational education and associated 
quality assurance mechanisms within the continuing 
and post-secondary sectors. With the proliferation of 
post-secondary education programmes and qualifica- 
tions, the Hong Kong OF ensures that education and 
training providers, employers and the public all under 
stand the range and different types of qualifications, as 
well as contributing to nurturing the core skills and com- 
petences of individuals and promoting lifelong learning 
across all sectors of society. The Hong Kong OF acts as 
a unified system for the recognition of quality assured 
qualifications for continuing education and post-second- 
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Table 4-3. NOFs and alternative progression pathways 
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DISTINCTIVE ALTERNATIVE COUNTRIES 
PROGRESSION PATHWAYS 


Continuing and post-secondary and employment 
pathways to quality-assured qualifications 


Pathways between VET and higher education 


Pathways between primary and VET 


Pathways from non-formal education to formal basic 
education 


Certification-oriented second-chance qualification 
programmes for/in the informal sector 


Source: Author. 


ary education, including all learning outside the formal 
education and training system. In this way NOFs are 
able to coordinate the diversity of learning already taking 
place outside the conventional system. In countries such 
as India, Malaysia, Maldives and Mexico learning activi- 
ties have increased substantially over the past 20 years 
facilitated by distance learning, open learning, special 
entry schemes, credit transfer arrangements and recog- 
nition of prior learning and other strategies the develop- 
ment of lifelong learning for adults. 


Countries such as Australia, India, New Zealand and Sri 
Lanka — and, more recently, Bhutan - have comprehen- 
sive NOFs that support the development of pathways 
between VET and higher education programmes, provid- 
ing access and progression to higher education as well 
as enhancing their value/currency in the labour market. 
Australia has credit transfer arrangements for workplace 
learning, providing means of progression to higher edu- 
cation programmes. The Bhutan qualifications framework 
(Figure 4-2) foresees progression for individuals from cer 
tificate level to the diploma level. 


Australia, Hong Kong (SAR China), India, Malaysia, 


Maldives, Mexico, New Zealand, Singapore, Thailand 


Australia, Bhutan, India, New Zealand, Republic of 


Korea, Sri Lanka 


Bhutan, Botswana, Ethiopia, Gambia, Ghana, India, 
Kenya, Mauritius, Namibia, Nepal, Seychelles, Uganda 


Bangladesh, Gambia, Mauritius, Philippines, Thailand 


Afghanistan, Bangladesh, Gambia, Ghana, India, 


Mauritius, Mexico 


4.6. Micro level: individuals demonstrate learning 
and competences 


Identification of contextual factors at the policy level, the 
key elements of the different ways organising education 
and training systems in different countries, and the pos- 
sible RVA routes and flexible pathways leading to quali- 
fications outlined above all provide the implementation 
environment for lifelong learning systems of quality. The 
next stage Is to concentrate on the learning dimensions of 
recognition, validation and accreditation processes as they 
affect the end-user. At the micro level, we concentrate on 
defining six core factors that need to be taken into con- 
sideration in organising a quality lifelong learning system, 
keeping in mind the usefulness of RVA for the end-user. 
These do not indicate how RVA should be implemented at 
the local level; they are initial indicators that make up the 
framework for organising quality of RVA at the individual 
level. Some examples are shared for illustrative purposes 
to help advance the discussion. 


4.6.1. Ownership and acceptance of local level stakeholders 

The specific priorities of the RVA system in a country must 
be identified not only from a recognition policy perspective 
but also by taking into account the roles of those involved 
at the local level: learners, employers, employees, training 
providers, trainers, educational institutions, NGOs, public 
and private organisations, representatives of trade unions, 
small, medium and micro enterprises, adult associations 
and youth organisations. Local needs should be compared 
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Figure 4-2. Progression pathways from VET to higher education in Bhutan 


Selection & bridging 


arrangement 


Selection & bridging by 


articulation 


Basic education 


Source: Cedefop, ETF UNESCO and UIL, 2017, Vol. 2. 


against macro level needs as well as related to the existing 
education and training institutions and providers and other 
sectors of the lifelong learning system. This process ena- 
bles local needs to be identified but also aims at securing 
substantial acceptance and ownership through the partici- 
pation of those involved. 


Training providers and staff of the National Institute of 
Open Schooling (NIOS) in India are directly involved in 
serving underprivileged and low-educated groups in so- 
ciety who have not completed eight years of school (the 
minimum to gain entry into a vocational training institute) 
and who lacked the basic literacy and numeracy skills 
necessary to enter the lowest level of the Indian skills 
qualifications framework. This target group is being given 
access to competence-based courses at prevocational 
levels in the qualifications framework through the NIOS. 
In the Netherlands, local stakeholders apply validation 
as a tool to tackle the economic crisis and target young 
unemployed persons lacking level 2 vocational qualifica- 
tions, those who are at risk of losing their jobs, or those 
who need to achieve mobility on the labour market. In 
Botswana and South Africa recognition serves to allow 
adults to participate in adult basic education and training 
(ABET) upon becoming literate. 


Entry requirements will 
apply for selected courses 
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RVA without stakeholder input will not be trusted. In the US, 
having at least one agency in the region supporting or encour 
aging the adoption and use of RVA methods has been high- 
lighted in a study conducted by the Council for Adult and Ex 
periential Learning (CAEL) (Klein-Collins, 2010). According to 
the purposes of RVA, representation from the world of work, 
and youth and adult organisations is required in developing 
RVA processes, methodologies and assessment criteria. 


In many countries local stakeholders need to be made 
aware of the recognition schemes and their benefits among 
potential users, including citizens, businesses and their em- 
ployees, education and training providers, voluntary associa- 
tions, and social partner organisations in the labour market. 
Mauritius is focusing on communication strategy to expose 
major stakeholders to international RVA best practice. 


An evaluation study from Denmark shows that the spread 
of RVA varied from institution to institution. Many institu- 
tions (education) have no RVA activity. Some of the bar 
riers are in lack of awareness of RVA; others deal with 
financing and connecting skills development to formal ed- 
ucation. Raising awareness for making the transition from 
system level to the user level or training provider level is 
an important quality issue. 
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For quality assurance of institutions and training providers, 
institutions must develop transparent guidelines to ensure 
quality, validity and reliability. In Australia and New Zealand, 
only registered training organisations that fall under the 
quality assurance framework of their NOFs also undertake 
recognition and validation of non-formal and informal learn- 
ing. The New Zealand Qualifications Authority (NZOA) has 
a policy that education providers use. 


Not only must a regulatory framework be in place at nation- 
al level to link recognition practices to NOFs, but It must 
also gain the acceptance of all stakeholders. For instance, 
in Mauritius RPL has been accepted by employers since it 
provides them with qualified and well-motivated person- 
nel. By contrast, in Hong Kong SAR, China, it appears that 
the government overestimated worker interest in further 
education, at least as presented by labour union represent- 
atives, and underestimated their insistence on using qual- 
ifications frameworks for job security and improving wage 
levels. It also lacked foresight in anticipating employers’ 
strong opposition to the use of potentially unverified work 
experience rather than assessed skills and competences. 
As of today, RPL and its pace of implementation is a mat- 
ter to be solely determined by the industries concerned 
(so far 22 industries have Joined the OF covering about 53 
per cent of the total labour force in Hong Kong SAR), and 
there is currently in-depth discussion to reach a consen- 
sus before proceeding with its implementation. However, 
Hong Kong sees RPL not as a stand-alone practice but in 
relation to other instruments such as credit accumulation 
and transfer (CAT), which takes into account the total time 
likely to be required by an average learner in all modes of 
learning, including attendance in classes, self-study, online 
learning, practical learning, and examination. The develop- 
ment of NOFs is expected to facilitate CAT arrangements 
between sectors and training providers by providing a uni- 
fied platform and common benchmarks. 


4.6.2. Resources and time 

Since the end-user and recognition processes at the local 
level form the focus of our approach, RVA must be appropri- 
ately resourced; this is the only way to address both qual- 
itative and quantitative RVA needs. Necessary resources 
include rooms and assessment centres, the dissemination 
of information, guidance and counselling, identification and 
documentation of evidence, target group specific tools and 
materials, and appropriately trained RVA staff. Norway has 
found that specific assessment tools at upper secondary 
level for adults include dialogue-based methods, portfolio 
assessment and vocational testing. Attention is paid to en- 
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suring the capacities of assessors and guides to interpret 
standards of relevant qualifications, curricula, objectives and 
admission requirements of the education programme in 
question, or workplace-specific competence demands. The 
Nordic model for quality in validation (Grunnet and Dahler, 
2013) includes three perspectives on quality resources: or 
ganisation quality entails developing a holistic approach for 
institutions to work with RVA, as well as the development 
of evaluation, feedback mechanisms and improvement in- 
itiatives; assessment quality consists of using distinct cri- 
teria, methodologies, and establishment of evaluation and 
documentation practices; procedural quality consists of the 
distribution of responsibility and roles, and clear informa- 
tion, through website presentations, brochures and docu- 
ments. The model is holistic, including all staff engaged in 
validation activities as practitioners, as well as dynamic and 
flexible, meaning it can be used in different institutional and 
sectoral contexts which may differ from country to country. 
In addition, the Nordic model is linked to eight factors in 
relation to which practitioners are free to decide how to 
use them and moderate them in their own contexts. These 
eight factors are information, preconditions (or regulatory 
framework), documentation, coordination, guidance, map- 
ping, assessment, and follow-up. 


National qualifications authorities are prime movers of RVA 
in many countries but there remains the challenge of how 
to equip education providers and training institutions for 
them to initiate the RVA exercise and offer opportunities 
to potential candidates. Bangladesh, with the assistance of 
International Labour Organisation (ILO), has plans to imple- 
ment recognition of non-formal and informal learning as an 
additional tool that will be integrated into training agendas 
of providers. This change will require competent assessors 
to carry out a diverse range of assessment methods to com- 
pare a student's existing competences with unit standards 
and qualifications in the frameworks. The organisation and 
the planning process will also need to incorporate issues 
such as industry needs as identified by skills committees; 
initial training of assessors and the sustainability {XE "sus- 
tainability: of the process of training assessors’ \f "subject'} 
of this process; procedures for the registration of providers; 
maintenance of a central register of qualifications; audit and 
moderation functions; and portfolio development. 


In South Africa, the take-up of RVA is limited by staff and 
resource shortages; projects have been developed only in 
a few sectors. Costs to individuals and education systems 
for information and guidance, assessors and awarding 
bodies are a further challenge. 
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Good practice from Denmark shows that arrangements for 
the collaboration of assessors across institutions provide 
good opportunities to discuss which tools are relevant. 
This collaboration ensures some alignment in the tools in- 
stitutions used in an RPL. In addition to knowledge sharing 
and collaboration, benefits could include a common ma- 
terial database and a manual for conducting RPL. Another 
challenge is follow-up on the outcome of the RPL through 
organisation of courses subsequent to the RVA. This is of- 
ten difficult for institutions and training providers. 


In Sweden, getting resources for RVA processes often 
depends on getting a commission to do validation from 
the public employment office (PEO) (Aagaard et al., 2017). 
Resources and time are preconditions. Getting a commis- 
sion depends on demand and on procurement processes 
where different validation institutions compete to be a pro- 
vider for the PEO. Quality includes developing information 
to candidates before the validation process; continuing 
professional development (CPD) for those who work with 
validation; improving the mapping for candidates for new 
arrived refugees; and improving quality of assessment in 
cases where the candidate lacks communication skills, but 
possibly has the vocational skills to be assessed. These are 
all resource-rich and time-consuming activities. 


4.6.3. Assessment practices and expertise 

Some authors have highlighted the tension between qual- 
ifications quality assurance and the lifelong learning goals 
of broadening access to adults and early school leavers. 
Comyn (2009) citing the Asian Development Bank (2008), 
suggests that NOFs may not be as relevant where the 
main problem is insufficient access to skills as opposed 
to improving the quality of assessment. Castejon (2013) 
also highlights that assessment is the core of qualifica- 
tions frameworks. Governments need to understand 
that the shift to an outcomes-based approach has many 
implications for assessment, validation and certification, 
and learning and teaching strategies. Traditionally, assess- 
ment has been based on requirements of, and expected 
performance in, formal education and training. This pro- 
cess however, is not suitable in a lifelong learning system 
that recognises diverse learning paths. In New Zealand, 
for example, assessment generally includes recognition 
of all learning from formal, non-formal and informal set- 
tings, against designated current learning outcomes or 
standards, which make up the qualifications. In Australia, 
workplace-learning assessment includes assessment of 
formal, non-formal and informal learning. By contrast, in 
developing countries such as Bangladesh, most informal 
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and non-formal workplace learning has not met a quality 
assurance requirement such as accreditation and is not 
recognised through any credit transfer arrangement. This 
situation is in contrast with that in Australia, where credit 
transfer arrangements exist even for workplace learning. 
In this case, non-formal learning is structured to meet for 
mal objectives and result in credits and a qualification; it is 
therefore viewed as a formal learning and falls under the 
jurisdiction of formal quality assurance processes. 


Assessment and validation of non-formal and informal 
learning should usually be a quality assured process as 
it results in the recognition of individual's knowledge and 
skills. The underlying principles of recognition, validation 
and accreditation (RVA) according to the UNESCO guide- 
lines include equity, the equal value of outcomes from 
all learning, and quality in the assessment and validation 
through procedures that are reliable, fair, transparent and 
relevant (UIL, 2012). Quality is essentially a matter of valid- 
ity and reliability in the RVA practice. Kirsten Aagaard, Per 
Andersson, Timo Halttunen, Brian Benjamin Hansen and 
Ulla Nistrup (2017) however argue that questions of what 
‘quality’ in RVA entails should not be taken for granted. In- 
stead, basic questions such ‘does the validation ‘measure’ 
or ‘assess ‘what Is intended’ or ‘how is this intention ne- 
gotiated and decided?’ need to be addressed. They argue 
that reliability and validity must not be taken for granted; it 
is rather a matter of negotiation of meaning, which could 
result in different situation-context-dependent conceptions 
of quality. These conceptions could include varying ideas 
on what knowledge and skills should be assessed, and 
how this could be done in the best way. 


National examples emphasise distinctive criteria for ensur 
ing quality of assessment and validation processes in cer 
tifying learning outside the formal system. 


First, the evidence of learning needs to be produced with 
more attention paid to assessing its validity and authen- 
ticity. Individual competences should be recognised irre- 
spective of where and how they were acquired, but with- 
out compromising the quality/standard of the education 
and training programme. The aim should be to assess the 
theoretical and the practical side of the trade. Evidence of 
learning is essential to assess credit-worthiness over and 
above the experience that shaped that learning (Wilbur et 
al. 2012). An example from Norway shows that methods 
of assessment at upper secondary level for adults include 
dialogue-based methods, portfolio assessment and voca- 
tional testing. The latter combines interviews and practice, 
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charting the learner's background, training, work experi- 
ence, language skills and objectives, and to observe his/ 
her skills in practice. The methods used need to ensure 
reliable assessment, inspiring confidence in the outcome. 


Second, quality assurance concerns the standards to 
which the evidence of learning is compared. These stand- 
ards should be directly comparable, preferably identical, to 
the standards applied in formal settings for the qualifica- 
tion. Care needs to be taken in ensuring these standards 
have been fairly interpreted. In France, assessment pro- 
cedures help candidates organise learning outcomes in a 
way that suits the standards of the relevant qualification, 
and prepare the candidates to meet the jury under the best 
conditions. In UK-Scotland, learning outcomes and skills 
gained through informal learning are mapped against the 
appropriate level of the Scottish credit and qualifications 
framework (SCQF). In Denmark, an important principle is 
that competence assessment should always be based on 
the objectives and admission requirements of the educa- 
tion programme in question. 


A third quality consideration for certifying learning out- 
side the formal sector concerns taking account of the 
candidates’ circumstances and the access they should 
have to reliable information, advice and guidance 
throughout the assessment process. The standards they 
must meet, the ways their learning is evidenced, the as- 
sessment process and the way assessed evidence is val- 
idated should all be clear to the candidate if the outcome 
of the process Is to be fair and trusted. At the same time, 
Aagaard and colleagues (Aagaard et al., 2017) emphasise 
acknowledging the two faces of quality in RVA: ‘On the 
one hand flexibility, individualisation, and judgement are 
central concepts. This perspective begins from an inten- 
tion to give recognition to individual knowledge and skills 
that have been developed in varying ways, and in differ 
ent contexts, thus probably situated in specific practic- 
es. ... On the other hand, standardisation, reliability, and 
measurement are central concepts. This is important 
where the results have to be comparable, for example, 
as a basis for fair selection processes in relation to high- 
er education or recruitment for apposition in the labour 
market’ (Ibid. p. 2). 


Fourth, quality needs to be seen in relation to the out- 
comes-based qualifications. The use of learning outcomes 
enables teaching and learning to be separated from as- 
sessment, allowing learners to be assessed differently 
according to their learning pathways: formal, non-formal 


VOLUME | — THEMATIC CHAPTERS 


or informal. For example, an increasing number of learners 
undertake adult education online or through work-based 
learning to acquire the competences needed to gain a 
qualification; these routes require different assessment 
methods and tools. They need to depart from the tradition- 
al final exam to encompass new forms of assessment that 
include practice-based assessment tasks requiring obser 
vation within a simulated or real context, evidence-accu- 
mulation (portfolio), and evaluation of real life practice. Be- 
cause these more varied assessment methods are now 
being used in relation to outcomes-based qualifications, 
more attention is being paid to assessment standards — 
including assessment criteria, procedures, guidelines and 
minimum requirements — to ensure the validity and relia- 
bility of assessments. 


Fifth, the result of the assessment should be document- 
ed by issuing a full qualification (or a certificate of edu- 
cation) or a part qualification (or credits, or a certificate 
of competence). 


Sixth, different kinds of expertise and resources are 
needed to develop an effective assessment and recog- 
nition system. Recognition practitioners include individ- 
uals delivering information, guidance and counselling; 
those who carry out assessments; the teachers and 
managers of education institutions; workplace instruc- 
tors; employers, and a range of other stakeholders with 
important but less direct roles in the recognition pro- 
cess. Guides/facilitators offer information, guidance and 
counselling services to refugees with the aim of clari- 
fying procedures for the assessment so that individu- 
als become more aware of their own competences and 
are more motivated to further learning and to have their 
learning outcomes recognised. 


The role of RVA personnel is underestimated in many 
cases. Only well-trained RVA personnel can manage 
meaningful recognition processes and communicate 
clearly about what the recognition of non-formal and 
informal learning really means. Nor can such skilled 
personnel be replaced by technology or online learning 
packages: any materials need guidance from RVA staff 
in using them. It is necessary, therefore, for RVA per 
sonnel to be adequately trained and employed in as- 
sessment centres and education Institutions, social en- 
terprises, workplaces and public employment centres. 
Existing RVA staff would need to have experience in 
workplaces as well as didactic and pedagogic aspects 
to their training. 
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RVA expertise should contribute in: 


(a) furthering the development of RVA practices; 

(b) broadening understanding of assessment and evaluation; 

(c) increasing knowledge of recognition processes through 
professional development opportunities; 

(d) using a variety of assessment methods and tools; 

(e) reviewing non-formal learning programmes and courses 
for continuous improvement; 

(f) communicating the vocabulary and concepts of recogni- 
tion to make RVA a part of acommonly accepted practice. 


Developing certificate courses for practitioners working in 
non-formal learning and in recognition of individual com- 
petences could help to improve the quality of teaching, 
guidance and counselling as well as help managers to 
coordinate non-formal learning and recognition processes 
(Austria). The certificates need to be positioned at a cer 
tain level of the qualifications framework and modules can 
be prepared for the formal diploma programme offered ei- 
ther in colleges, or in continuing education departments 
of universities. In Portugal, professionalisation is sought 
through sharing practices, knowledge and experiences 
among teachers and trainers who carry out adult learning 
programmes and undertake validation assessments. Good 
practices from Denmark show that arranging collaboration 
among assessors across institutions provides good oppor 
tunities to discuss which tools are relevant, ensuring some 
alignment in use in an RVA. There is also potential bene- 
fit in a common material database and a manual for con- 
ducting RVA. A common language to describe the subject 
matter of RVA and tools is necessary to reach significant 
numbers of individuals. 


4.6.4. Progression 

Many RVA candidates are those whose knowledge, skills 
and competences are still in the process of being devel- 
oped, so RVA should be conducted as part of an education 
and training programme or a preparatory bridge course. It 
is crucial to make participation in education and training 
or preparatory courses as attractive to them as possible, 
as is for instance the case in Finland. All RVA processes 
should therefore centre on assessment in terms of an ex 
tent to which they motivate participation in a further ed- 
ucation and training. It is important that when education 
and training programmes are being put in place, the State 
focuses particularly on progression. State bodies should 
ensure that that learning processes end in an assessment 
of knowledge, skills and competences that are then rec- 
ognised and certified according to agreed procedures; this 
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is the only way of ensuring transparency of the standard 
achieved by individual participation in RVA. The certification 
should ensure transitions into the labour market for individ- 
uals and groups of individuals who are not yet integrated 
into the employment system, as well as access to general 
education. In Denmark, an evaluation study (Andersen and 
Laugesen, 2012) showed that while RVA is primarily used 
to give access to the education programmes that institu- 
tions provide, most candidates apply to gain recognition of 
competences for direct use in the labour market 


In developing countries where vocational education and 
training are of low status, it is crucial that more substantial 
training courses are certified in a way that also ensures ac- 
cess to general education for the trainee concerned. This 
will be important to boost the status of vocational educa- 
tion and training and cater to a greater parity of esteem 
between formal and non-formal learning. Access to broad- 
er skills through general education is particularly important 
in the context of rapid economic and societal changes. 
These changes increase the importance of personal de- 
velopment, while reducing the importance of task-specific 
and narrowly defined instrumental knowledge and skills, 
as Is for instance the case in New Zealand. Progression 
pathways in the US are called ‘certification crosswalks’. 
These include College credit for what you already know, 
a project developed by the Council for Adult and Experi- 
mental Learning (CAEL) designed to bring prior learning 
assessments to scale, and increase the number of adults 
who would benefit from access to college education pro- 
grammes (Ganzglass, Bird and Prince, 2011). 


In Canada, progression through access to formal qualifi- 
cations remains the key aspect of prior learning assess- 
ment and recognition. However, opening up access and 
progress in skilled and professional occupations is now 
reported as the key issue. 


4.6.5. Encouraging employer and training provider involvement 
Employers, who ultimately, are the users of skilled staff, 
need to be consistently encouraged to become involved 
in the training/recognition processes as this is a way of 
ensuring that skills development builds on actual practice 
and that skills reflect the real world. Employers should 
be involved in shaping recognition processes and training 
courses. They may find it helpful to use recognition to be- 
come more familiar with the stock of knowledge, skills and 
competences in their enterprise. In Mexico, workers ob- 
taining certification of competences is considered a good 
proxy for increasing worker and the firm productivity and 
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reducing worker turnover (Garcia-Bullé 2013). In Mauritius, 
employers are encouraged to invest in training those with 
very low skills, who need to be brought into the productive 
economy. The Mauritius Qualifications Authority (MQA) is 
making concerted efforts to implement RVA to support 
workforce development in certain industry sectors (spe- 
cifically tourism, financial services, real estate, information 
and communication technology (ICT) and seafood). Em- 
ployers understand the role of RVA in supporting a highly 
skilled workforce and they contribute to MOA fees for this. 
Well-established companies are also sponsoring RVA can- 
didates. Currently, 19 industry training advisory commit- 
tees are generating NOF qualifications in all TVET sectors 
of the Mauritian economy. While such qualifications are 
offered by both public and private providers, a centralised 
awarding body awards the NOF qualifications. In Mauri- 
tius, RVA and the NOF coexist in a symbiotic relationship, 
where the former is directly linked to the qualifications out- 
comes and smooth transition of many learners to the NOF 
is possible (Allgoo, 2013). 


Training providers can enhance learning in different ways, 
both formally and non-formally. A positive learning envi- 
ronment and policies should be developed by providers 
to create more learning opportunities and enlarge learner 
and worker abilities. A more active learning model should 
be promoted and partnerships with organisations such as 
commercial firms, education providers, youth organisa- 
tions or social enterprises, should be improved to expand 
opportunities for students, the young and adults. 


Employers can support the education sector, adult, and 
community learning sectors in two ways: increasing the 
number of business partnerships and providing further job 
placements. They can also contribute to student assess- 
ment, providing feedback on learning at the workplace 
helping evaluate the efficiency of joint curricular activities 
and workplace learning (Leung and McGrath, 2010). 


However, the concerns and needs of companies need to 
be taken into account in RVA. They must receive a return 
on their investment. In-company training and recognition 
initiatives should be designed in such a way that this a 
clear advantage to companies in terms of their cost/benefit 
of engaging in such activities (Pilz 2017). 


4.6.6. RVA-benefits for the individual worker and learner 

Recognition of an individual's learning and competences 
is important for quality. Recognition does not only mean 
official recognition but also recognition by the person 
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whose knowledge has been validated. To be admitted 
to an education institution or to be recruited and em- 
ployed, would mean recognition could be important for a 
strengthening self-confidence. RVA has different purpos- 
es, which are important for deciding what quality is in a 
certain RVA context. It is usual to identify a formative or 
summative goal. Formative RVA Is intended to diagnose 
prior learning, forming the basis for further learning; 
quality here means providing the best possible founda- 
tion for further learning. Summative validation is typically 
undertaken by summing up the results of learning and 
competences in relation to certain criteria. Ideally both 
forms of assessment should make individuals aware of 
their learning and competence, strengthen self-confi- 
dence and guide people to lifelong learning. 


RVA must have real benefits for specific target groups such 
as refugees and migrants as well as the low-skilled, semi- 
skilled and skilled workers and learners who are already in 
employment. In certain sectors of the labour market, the 
demand for workers with the requisite knowledge, skills 
and competences is growing faster than the supply, often 
because the formal education and training system is not 
sufficiently responsive. 


In the case of low-skilled refugees in Sweden, the valida- 
tion model has a clear focus on quality through an extensive 
process of identifying and validating candidates’ compe- 
tences, including practical work tasks, and the opportunity 
for ‘topping-up’ with context-specific skills that are lack- 
ing. Employability is an important benefit for employees as 
it is also a matter of establishing credibility and currency in 
relation to the industry. Initial mapping should show that 
the candidates could fulfil at least half of the requirements 
for the further validation process to be meaningful; if not 
the main alternative is to take the full training programme. 


Consideration needs to be given to how skilled workers 
can be appropriately employed so that their competences 
can be used meaningfully and in way that will motivate 
them (Pilz, 2017). Factors playing a role here include not 
only technical skills but also aspects such as computers, 
health, and safety at work. Even more important, howev- 
er, IS Increasing remuneration. Demand for training, and 
the likelihood that training activities will bring success, 
depends on there being a long-term monetary advantage 
to the individual employee in acquiring skills. State agen- 
cies and social partners should work towards a regulatory 
framework that prevents discrimination and market distor 
tions. In Germany, RVA features in collective agreements, 
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giving greater security to individuals who have acquired 
skills through informal and non-formal learning in recog- 
nised apprenticeship trades. Similarly, provisions exist 
within the German public sector for scaled remuneration 
on the basis of work experience and length of service. |n- 
dividuals can enrol in training programmes provided that 
they have a minimum of practical experience, with industry 
training agencies providing leadership in the design and 
development of RVA processes. 


4.7. Conclusions 
This chapter demonstrated that a quality lifelong learning 
system does not only depend on the existence of policies, 


frameworks and standards, but also on the relevance of 
delivery and quality of qualification and recognition ar 
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Summary 


Validation of non-formal and informal learning, or the 
efforts to make visible and value the learning taking 
place outside formal education, is gradually becoming 
an integrated part of national education and training and 
lifelong learning arrangements in Europe. While most 
countries now officially state that they aim for the intro- 
duction of national validation systems, this is not always 
translated into practical arrangements on the ground 
giving citizens access to validation. Policy efforts to in- 
troduce validation date back more than three decades. 
The purpose of this chapter is to discuss the extent to 
which the values underpinning validation have been ac- 
cepted and/or internalised at national and European lev- 
els. While the chapter contributes to better understand- 
ing of the history of this policy field, it also illustrates 
how national and European level policy initiatives have 
interacted over time. 


Keywords: validation of non-formal and informal learning; 
national qualifications framework (NOF); European qual- 
ifications framework (EQF); non-formal learning; informal 
learning; prior learning assessment; learning outcomes 


5.1. Introduction 


Validation of non-formal and informal learning can be 
defined as the process that makes visible and gives 
value to the learning taking place outside formal edu- 
cation. It has grown in importance and visibility over 
the past few decades. Stakeholders at national as well 
as European level have argued consistently that valida- 
tion can play a key role in opening up education and 
training systems to the learning taking place at work 
and during leisure time. Validation is increasingly being 
related to the needs of groups at risk, arguing that it can 
support integration into the labour market and society 
at large (Souto-Otero and Villalba, 2015). This chapter 
looks into the development of validation in Europe over 
the past few decades and discusses to what extent the 


fundamental values underpinning validation have been 
accepted and internalised. There is also reflection on 
the interaction between national and European policies 
in this area. 


5.1.1. Validation: an issue of values 
The acceptance of validation of non-formal and informal 
learning into society implies acceptance of two main values: 


(a) all learning, irrespective of where and when it takes 
place, is valuable for the individual and for society; 

(b) formal learning needs to be complemented by validation 
of non-formal and informal learning to make visible and 
value the rich, diversified learning of individuals. 


These two values are closely interlinked and constitute 
what will be referred in this chapter as ‘the validation 
norm’. Actively promoting this norm means that the dom- 
inant position of formal education and training systems is 
challenged. The introduction of validation arrangements, 
reflecting that learning de facto takes place in a wide range 
of settings and institutions, implies questioning the ‘exclu- 
sive’ right of formal education and training to value and 
certify learning. 


For this norm to be accepted, and for validation to be- 
come an integral and effective part of national policies 
and practices, three main conditions have to be fulfilled. 
The first refers to the institutional setting of validation: 
are adequate laws, institutions and the relevant stake- 
holders committed to these values, allowing for long- 
term and legitimate implementation of policies? The 
second condition refers to resources and whether policy 
objectives are translated into concrete arrangements on 
the ground, giving citizens access to validation. Are the 
validation initiatives properly resourced, with well-trained 
professionals and adequate professional development 
opportunities? The third condition refers to methodolog- 
ical issues. Validation requires that the methodologies 
used for validation guarantee reliability and validity of the 
learning outcomes acquired. 
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A lack of trust in any of these three conditions undermines 
the principle that all learning is equal and that formal learn- 
ing needs to be supplemented with validation of non-for 
mal and informal learning. This chapter discusses the ac- 
ceptance and internalisation of the ‘validation norm’. The 
key questions we address in this chapter are: 


(a) have the values underpinning the promotion of validation 
been internalised by EU countries? 

(b) have the associated conditions for implementing valida- 
tion at national and European levels been addressed? 


The chapter starts by reviewing the recent history of valida- 
tion in Europe from its emergence in the 1980s and 1990s. 
A second stage is presented from 2002 to the adoption of 
the 2012 recommendation on validation. The third part of 
the chapter discusses the current situation and the extent 
to which we can speak of acceptance of the basic norm, 
or not at this stage. Analysis of this third stage is based on 
the data collected for the latest update of the European in- 
ventory on validation of non-formal and informal learning ("). 


5.2. The early days (1980-2002): emergence of a norm? 


Validation, as a separate policy field, starts to emerge dur 
ing the late 1980s. This does not mean that countries start- 
ed without prior experience in this area: several already 
operated with arrangements allowing individuals with rele- 
vant work experience to sit for exams, in effect awarding a 
qualification without attending classes. The so called Arti- 
cle 20' arrangement in Norway, in existence since 1952, il- 
lustrates this. These arrangements, however, were mostly 
seen as technical, ensuring the flexibility of (mainly) formal 
training, and not as policy initiatives in their own right. 


5.2.1. National developments 

The emergence of the ‘validation norm’ — or the acknowl- 
edgement that all learning, irrespective of the context 
in which it takes place, should be recognised — is close- 
ly linked to the introduction of learning outcomes and/or 
competences based education standards and curricula 
in the 1980s and 1990s. The shift to learning outcomes, 
focusing on what learners are expected to know, be able 
to do and understand, states that the same outcomes 
can be reached in different ways and by following a va- 
riety of pathways (including learning at work and during 
leisure time). The introduction of national vocational qual- 


('), www.cedefop.europa.eu/validation/inventory 
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ifications in the UK in the 1980s and the development of 
competence-based education in Finland in the 1990s both 
illustrate how national learning-outcomes-based approach- 
es trigger the development and introduction of validation 
arrangements. In the UK, the combination of learning out- 
comes (and modularised qualifications) resulted in arrange- 
ments including accreditation of prior learning (APL) and 
accreditation of prior experiential learning (APEL). In 1991, 
the National Council for Vocational Qualifications required 
that ‘accreditation of prior learning should be available for 
all qualifications accredited by these bodies’ (European 
Commission et al., 2008). 


The extent to which this shift to learning outcomes aided 
the introduction of validation arrangements varies. While 
the Finnish competence-based system has developed into 
a strong and integrated part of the national system, and 
is still Serving many individuals, the national vocational 
qualifications (NVQ) system has been limited in its ability 
to promote validation of non-formal and informal learning. 
The relative lack of progress in the UK may come down to 
lack of policy priorities, but may also be linked to the con- 
troversy around the quality and the relevance of the NVO 
system. Some of the criticism currently raised against the 
learning outcomes approach (and, implicitly, validation) 
seems to be linked to the particular (‘narrow’) way learning 
outcomes were defined for NVQs (Allais, 2016). So, while 
learning-outcomes-based standards and curricula are im- 
portant in promoting validation, they cannot do so alone. 


France was also a frontrunner among the countries that 
pioneered validation. The bilan de compétence was es- 
tablished in 1985, supporting employers and employees 
in identifying (making visible) competences acquired at 
work. From 1992 vocational certificates (certificat d’ap- 
titude professionelle) could be obtained (to various de- 
grees) on the basis of assessment of non-formal and 
prior learning, and in 2002 legislation was adopted estab- 
lishing a comprehensive national framework for valida- 
tion (validation des acquis de I’expérience (VAE)). Nordic 
countries increasingly focused on developing legislation 
and institutional solutions allowing for validation. Nor 
way, as a part of the 1999 Competence Reform, carried 
out an extensive three-year experimental scheme to de- 
velop and test the elements necessary for an operational 
national system on validation (VOX, 2002). This led to the 
setting-up of operational validation schemes from the 
early 2000s, addressing vocational education and train- 
ing in particular. In the Netherlands, the commission on 
Erkenning Verwoven Kwalificaties (EVK) developed rec- 
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ommendations to establish a system for validation that 
was then tested in some sectors (construction industry 
and childcare). Also in Switzerland, the association CH-Q 
Swiss qualification programme for job careers started to 
develop methodologies for assessing learning acquired 
outside the formal system in 1999. 


All these countries can be considered ‘entrepreneurs’ 
in the sense that Finnemore and Sikkink (1998) use the 
term. They ‘pave the way’ for turning validation into a 
visible policy priority, stressing the values underpinning 
the ‘validation norm’. Stakeholders in these countries fo- 
cused on the need to make education and training more 
flexible and to open up new routes for acquiring formal 
qualifications. These initiatives were partly framed in the 
context of lifelong learning, notably by avoiding dead 
ends and seeking to strengthen permeability in education 
and training systems. 


5.2.2. European level 

Bjornavold (2000) and Duvekot, Schuur and Pulusse 
(2005) point to the 1995 Commission white paper on 
teaching and learning (European Commission, 1995) as 
the first explicit effort to promote validation at Europe- 
an level. The white paper emphasised the importance 
of recognising competences acquired outside formal 
education and paved the way for extensive testing and 
experimentation to be financed through European pro- 
grammes such as Adapt, Leonardo da Vinci, Socrates, 
Equal. This experimentation, also supported by the work 
of Cedefop (Bjoarnavold, 2000 and 2002), focused on the 
development of methodologies for validation, particular 
ly testing the practical feasibility of the approach. While 
only in a few cases leading to permanent arrangements 
for validation being set up, the European programmes 
played a key role in disseminating ‘the validation norm’ 
to countries, institutions and experts previously not in- 
volved in this area. 


The 2001 communication of the European Commission 
Making the European area of lifelong learning a reality 
(European Commission, 2001) gave further impetus to 
the role of validation of non-formal and informal learning. 
Rooted in the Delors (1996) declaration, UNESCO's 1972 
publication Learning to be (Faure et al., 1972) and the sub- 
sequent changes to the concept of lifelong learning (e.g. 
Rubensson, 2001; Jarvis, 2002; Villalba, 2006), the 2001 
communication emphasised the importance of learning 
throughout one’s life and across the life span (in formal, 
non-formal and informal settings). Rubensson (2003) has 
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argued that the communication places major emphasis 
on informal learning and gives the individual significant re- 
sponsibility in the management of their learning history. 
Thus, validation became a central element in the imple- 
mentation of lifelong learning policies. Around the same 
time, the European Union started developing transparency 
tools that would allow for better portability of skills and 
qualifications. The Copenhagen declaration of 29 and 30 
November 2002 launched the European strategy for en- 
hanced cooperation in vocational education and training 
(VET). The declaration established the need for ‘developing 
a set of common principles regarding validation of non-for 
mal and informal learning with the aim of ensuring greater 
compatibility between approaches in different countries’ 
(European Council and European Commission, 2002). 


5.3. Expansion of validation (2002-12) 


5.3.1. National developments 

At national level, the ‘entrepreneur’ countries, exemplified 
by France, Finland, Norway and the Netherlands, institu- 
tionalised their validation approaches during this period, 
notably by integrating validation arrangements into their na- 
tional education and training systems and by increasing the 
number of those being validated. Several other countries 
acknowledged the potential importance of validation and in- 
itiated systematic processes for Its implementation during 
this period; Portugal is the most notable example. The New 
opportunities initiative (2005) defined a national strategy, 
largely based on validation, to raise the qualification level 
of low-qualified individuals. Including a national system for 
recognition, validation and certification of competences 
(RVCC), the initiative established more than 400 centres at 
local and regional level and led to the award of more than 
300 000 certificates. Denmark established legislation in 
2007 on developing recognition of non-formal and informal 
learning across all education sectors and launched several 
initiatives with the aim of increasing its use. In Germany, 
the introduction of the ProfilPass (a tool to document peo- 
ple’s skills) in 2005 can be considered a first move towards 
a broader validation approach. In Spain, the Royal Decree 
1224/2009 on recognition of professional competences ac- 
quired through work experience established mechanisms 
for validating non-formal and informal learning in VET qual- 
ifications. Belgium, Estonia, Iceland, Ireland, Luxembourg 
and Slovenia also exemplify the introduction of legislation 
and administrative procedures for validation, although the 
degree of practical implementation varies. Other countries, 
such as the Czech Republic and Lithuania, also initiated de- 
velopments during this period. 
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5.3.2. European level 

European programmes play a key role in testing solutions 
and disseminating experiences of (and attitudes to) val- 
idation. In the decade 2002-12, European Social Funds 
played a crucial role in supporting the setting up of valida- 
tion arrangements in ‘new’ countries, exemplified by the 
new opportunities programme in Portugal. Programmes 
such as Leonardo da Vinci, Grundtvig and Socrates (later 
the lifelong learning programme) continue support to test- 
ing and piloting, overall supporting several hundred pro- 
jects. These projects were largely triggered by the policy 
objectives on validation included in the lifelong learning 
initiatives, the Copenhagen process and in the Bologna 
process. This interaction between European policy initia- 
tives, European programmes and national developments 
is of key importance in understanding developments dur 
ing this period. Further, adoption of instruments such as 
the EOF and increased attention to the learning outcomes 
principle can be seen as an indirect (and important) sup- 
port to ‘the validation norm’. The broad implementation of 
learning outcomes in all sectors of education and training 
and in most European countries during the decade (Cede- 
fop 2009b, 2016) means that the conditions for opening 
up qualifications to a wider range of learning pathways 
were being addressed. 


In 2004, the Council adopted a set of conclusions on Com- 
mon European principles for the validation of non-formal 
and informal learning (Council of the European Union, 
2004). These principles were formulated at a high level of 
abstraction and identify issues and conditions critical to 
the implementation of validation (Cedefop, 2009a). This 
was followed up by the first European inventory on valida- 
tion (Cedefop, Colardyn and Bjornavold, 2005), providing 
an overview of national developments and arrangements 
followed by other editions (Souto-Otero, McCoshan and 
Junge, 2005; Souto-Otero, Hawley and Nevala, 2008; Eu- 
ropean Commission, Cedefop and GHK, 2010). Work on the 
first set of European guidelines on validation of non-formal 
and informal learning was also started (Cedefop, 2009a). 
Both the inventory and the guidelines support nation- 
al implementation of validation in Europe and have been 
widely disseminated and used. The 2004 Council decision 
on a single community framework for the transparency 
of qualifications and competences in Europe (Europass) 
(European Parliament and Council of the European Union, 
2004) can also be said to support the ‘validation norm’. The 
Europass CV is explicitly focused on the identification and 
documentation of learning outcomes in different contexts, 
including those acquired through non-formal and informal 
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learning. The 2008 adoption of the European qualifications 
framework (EOF) is of key importance to validation. The es- 
tablishment of the EOF triggered the introduction of learn- 
ing-outcomes-based national qualifications frameworks 
across the continent. The long-term effect of this, directly 
influencing validation, is the more systematic promotion 
of learning outcomes at national level. When countries are 
referencing their NOFs to the EOF, the role of validation is 
explicitly addressed. 


While initially largely focusing on vocational education and 
training, other areas and sectors were gradually being in- 
cluded in the policy discourse. In higher education, the 
communiqué of the conference of European ministers re- 
sponsible for higher education held in Leuven and Louvain- 
la-Neuve on 28 and 29 April 2009 (European Council, 2009) 
expanded the Bologna process and recognition convention 
to include recognition of prior learning, including non-for 
mal and informal learning. In 2011 the Council conclusions 
on the modernisation of higher education also called on 
Member States to develop clear routes into higher educa- 
tion from vocational and other types of education, as well 
as mechanisms for recognising prior learning and experi- 
ence gained outside formal education and training. 


Validation also gained importance for the young. In 2006 
a resolution of the Council invited Member States to en- 
able the identification of competences acquired through 
non-formal and informal learning for young Europeans. The 
resulting Youthpass (as the Europass) supports the docu- 
mentation of all forms of learning and promotes transfer of 
learning as well as transparency of qualifications (European 
Parliament and the Council of the European Union, 2006). 
This was followed up in 2009 by the renewed framework 
for EU cooperation on the young that places non-formal 
learning and Its validation at the core of youth initiatives. 


In this second phase we can see gradual expansion of 
validation objectives from the pioneering countries to 
an increasing number of ‘newcomers’. In the pioneering 
countries, validation was becoming more institutional- 
ised; although not necessarily always fully developed, the 
arrangements were becoming increasingly established 
within the institutional framework. In other countries, new 
legislation was developed to try to boost validation prac- 
tices, in many cases within the development of national 
qualifications frameworks. However, initiatives were still 
limited to specific sectors and often lacking full-scale im- 
plementation. While the number of countries working with 
validation increased, full-scale, comprehensive implemen- 
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tation was largely lacking. Validation was taken forward in 
a fragmented way, lacking overall coordination. Practices 
still remained confined to specific sectors with no relation- 
ship to other practices. However, some critical conditions 
for development were created, notably through the inten- 
sified role of NOFs and the more systematic focus on the 
implementation of learning outcomes. 


5.4. Internalisation of the validation norm? 
(2012- present) 


In 2012, following an open consultation, the European 
Council adopted the recommendation on validation of 
non-formal and informal learning (Council of the Europe- 
an Union, 2012). Different from previous occasions, where 
validation was treated as a part of broader initiatives, the 
2012 recommendation establishes validation as an inde- 
pendent policy instrument, relevant for policy develop- 
ment in several areas. The public consultation preceding 
the recommendation demonstrated that validation was 
considered increasingly relevant and important by most 
EU countries. The timing of the 2012 recommendation is 
important: following the financial crisis in 2008-09 a major 
ity of EU countries faced serious problems linked to unem- 
ployment, redirection of people's careers, marginalisation 
of social groups and a general rise in poverty and social ex- 
clusion. Seen from this perspective, the perception of val- 
idation changed. While previously seen by many as an in- 
strument to increase the flexibility of formal education and 
training (open up qualifications to non-formal and informal 
learning), countries now saw validation as a way to support 
integration of groups at risk and ‘reskill’ unemployed work- 
ers. Validation changed from being a tool relevant to the 
education and training sector to an instrument of interest 
to labour market and social policies. The adoption of the 
recommendation is not an Isolated initiative at European 
level, but reflects a changing political and economic reality, 
requiring responses at national level. 


5.4.1. European level: the 2012 recommendation, defining ‘the 
validation norm’ 

The recommendation clarifies the concept of validation, 
establishing a common understanding of what it is. The 
concept had remained difficult to define, with different 
terms used in different countries and contexts: valida- 
tion of non-formal and informal learning, prior learning 
assessment, recognition of prior learning, certification 
of learning, accreditation and validation of experiential 
learning and more. All these terms are related, but ad- 
dress slightly different aspects and ideas. The recom- 
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mendation provides a definition that can serve as an 
umbrella for all these existing, related terms. It defines 
validation as ‘...a process of confirmation by an author 
ised body that an individual has acquired learning out- 
comes measured against a relevant standard’ (Council 
of the European Union, 2012, p. 5). This process, accord- 
ing to the recommendation, consists of four different 
phases: identification, documentation, assessment and 
certification. The four stages permit much needed flex 
ibility in the validation concept. They make it easier for 
countries to adapt and accept the norm and to articulate 
the concept to represent the complex different realities 
in which it operates. The definition also indicates that 
the process of confirmation is carried out by an author 
ising body that checks the learning outcomes an individ- 
ual has acquired, measured against relevant standards. 
The inclusion of an authorising body means that certain 
institutional structures have to be in place. This address- 
es one of the three basic critical conditions referred to at 
the beginning of the chapter: the necessary institutional 
setting for validation. 


The recommendation draws up a series of principles to be 
included when establishing validation arrangements. Build- 
ing on the 2004 principles (Council of the European Union 
2004), these are more concrete and better integrated into 
existing policy actions. Validation arrangements, for exam- 
ple, should be closely linked to and integrated with guidance 
and counselling services, they should be closely linked to na- 
tional qualifications frameworks, and thereby to the EOF as 
well as working together with existing credit systems. Mem- 
ber States are also asked to make use of existing transparen- 
cy tools, especially Europass and Youthpass. The standards 
used for validation are meant to be the same as, or equivalent 
to, those of formally acquired qualifications. In this, the shift 
to learning outcomes described above becomes relevant as 
standards for validation cannot rely on time or place of learn- 
ing, but on learning outcomes irrespective of how they have 
been acquired. Other principles relate to the transparency of 
quality assurance and the provision for professional develop- 
ment to guarantee trust and reliability. 


The recommendation also establishes a body that over 
sees and is responsible for implementing the recommen- 
dation: the European qualifications framework advisory 
group (EOF AG). This gives Member States a forum for dis- 
cussion and exchange of views as well as a place for the 
norm to be further internalised. Giving responsibility to a 
specific body assures a certain degree of commitment and 
peer pressure in implementing validation initiatives. 
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The recommendation also gives an important role to the 
European guidelines (7) and the inventory on validation 
as tools to support its implementation, offering an extra 
platform for discussion and common understanding. The 
guidelines seek to clarify the conditions for implement- 
ing validation arrangements and can be regarded as the 
text in which the ‘validation norm’ is further defined and 
shared, providing the blueprint for the adoption, accept- 
ance and adaptation of the values and principles included 
in the norm. While the guidelines provide the principles 
and conditions to consider when implementing validation, 
the European inventory provides an overview of how vali- 
dation is actually being implemented across Europe. Sec- 
tion 5.4.2 presents data on national developments from 
the last inventory update to illustrate to what extent the 
‘validation norm’ is being actually adopted and internal- 
ised in European countries. 


5.4.2. The national level: confirming ‘the validation norm’? 

After the 2012 recommendation, validation arrangements 
were further institutionalised and an increasing number of 
countries developed possibilities for individuals to validate 
their knowledge and skills. The most recent edition of the 
European Inventory on validation (°) shows that in 2016, all 
countries now are developing validation and have put in 
place concrete arrangements in at least one sector of edu- 
cation and training. Table 5-1 shows the countries in which 
it is possible to obtain a full or part qualification, access, 
exemptions or provision of training specifications. 


The table shows that validation of non-formal and informal 
learning is becoming a common feature in European coun- 
tries. Validation arrangements are most common in voca- 
tional education and training and higher education, less so 
in general education. Outside the education and training 
sector, linked to the third sector, chamber of commerce, 


(*) The European guidelines, reviewed in 2015 after its first edition 

in 2009, were the result of a process of consultation with Mem- 

ber States and relevant stakeholders that lasted several years. 

They can be found at: http://www.cedefop.europa.eu/en/publica- 

tions-and-resources/publications/3073 

(8) The 2016 European inventory on validation of non-formal and infor 
mal learning consists of a series of country and thematic reports 
as well as a synthesis report summarising main results. It cov- 
ers 33 countries. There are three reports for the UK (England and 
Northern Ireland; Scotland; Ireland) and two for Belgium (French 
Community and Flemish Community). These regions are referred 
to and counted as ‘countries’ here and in the inventory. Countries 
include the 28 EU Member States, the EFTA countries and Tur 
key. The inventory addresses all aspect discussed in the European 
guidelines. It provides a detailed view of the different policies and 
practices on validation. http://www.cedefop.europa.eu/en/events- 
and-projects/projects/validation-non-formal-and-informal-learning/ 
european-inventory 
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public employment services and in private companies we 
increasingly see validation taking a hold. These initiatives, 
frequently focusing on identification and documentation 
of skills and competences, normally fall short of provid- 
ing certification and are rarely linked to national qualifica- 
tions systems. Only in a few countries, as with the bilan 
de compétence in France, do these arrangements operate 
on a large scale. 


Fragmentation —- a common feature of the period before 
2012 — remains a challenge for the current implementation 
of validation. The 2016 inventory, however, demonstrates 
that countries are working towards increased coherence 
across sectors. Around 60% of the countries under re- 
view have or are developing comprehensive systems for 
validation, while the other 40% have opted for a sectoral 
approach, in which the legal frameworks, strategies and 
policies are developed separately in different sectors. Irre- 
spective of the approach chosen — comprehensive or sec- 
toral— mechanisms for coordination across sectors are be- 
ing developed, with 20 countries reported to have in place 
an institution in charge of coordination. This coordination 
is mostly under the ministry or national organisation that 
oversees the different subsectors. 


The continuous development of NOFs, with most coun- 
tries moving into an operational phase, has also aided 
coordination. In Poland, for example, the Ministry of ed- 
ucation was given the role of coordinating the delivery of 
the integrated qualification system (IOS), which came into 
force on 15 January 2016. The IOS act describes all qualifi- 
cations awarded in Poland by authorised entities. It spec- 
ifies the requirements for bodies carrying out validation 
and certification as well as setting the principles for quality 
assurance in validation and its monitoring process. In 2010 
there were 12 countries in which an individual could ac- 
cess or obtain (part of) a qualification included in the NOF: 
in 2016 this is the case in 28 countries. 


Institutional structures, therefore, are increasingly ad- 
dressing the needs of validation. It is not clear, however, 
if this institutionalisation is being matched by an increase 
in human and financial resources. Determining the level of 
resourcing made available for validation is not straightfor 
ward. Many validation activities are embedded into train- 
ing provider budgets or form part of broader initiatives and 
specific projects, making it difficult to identify exactly what 
is used for validation. The 2016 inventory features a the- 
matic review of validation financing. The report concludes 
that policies on validation are not always accompanied by 
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Table 5-1. Possible outcomes of validation in one or more education sectors with validation arrangements 


AWARD OF PARTIAL/FULL FORMAL QUALIFICATION 


ANT, BEHill, (BG), Chl, CZ, DIE, IDX, IEIE, IES, (Fil, (ARI, IIE, WS, IM, WI), Way, LWW), IE, IMM, INIL, IN), IIL, IPI, IFO), SIE, Sil, WIR, WI (Beal), 


UK (S), UK (W) 


AWARD OF OTHER NON-FORMAL AWARD OF CREDITS 
QUALIFICATION/CERTIFICATE 


ANT, BIE-iii, GN, IDE, [EIL, IBS, (Fil, IIE, IS, 1), INIL, AL, SI, UI (S) 


AN, (BIER, Chri, IDE, IDK, [BIE ES, FF, In, IIE, IIS, IM, Lt, (ar 
EUR VAMIPNESNO SPIES isl Se UG(ESNI) WKS) 
UK (W) 


AWARD OF MODULES EXEMPTIONS 


Al, BE-fl, BE-fr, CH, DK, EE, ES, Fl, IE, 17, LU, LV, MT, NL, 


NO, PL, SI, UK (E&NI), UK (S), UK (WW) 


JN, BE=ill, IE; Cll, GZ, ID, [EIE, ES, [Fl InlU), UE, IS, Il, 
LU, LV, MT, NL, NO, PL, PT, SE, SI, UK (E&NI), UK (S), 
UK (W) 


ACCESSTO FORMAL PROGRAMMES TRAINING SPECIFICATIONS 


ANY, BIEAi|, (BEsiy, XG, rl, CZ, IDIE, IDIS, [EIE, IES, IIS, IIE, (Ll, |, 
LU, LV, MT, NL, NO, PL, SE, SI, UK (E&NI), UK (S), UK (W) 


NB: Multiple responses possible. 


Source: 2016 European inventory. 


a clear allocation of resources and European Social Funds 
are still a major source of income for initiatives. This was 
also one of the conclusions of the Cedefop conference in 
November 2016 (*). Participants pointed out that increasing 
validation structures was not necessarily link to adequate 
resourcing. This signals the temporary nature of validation 
arrangements as was the case before 2012, although it 
seems less severe than then. 


5.5. Conclusions 


The 2016 data show that most European countries have 
now accepted and internalised the ‘validation norm’ as de- 
fined at the start of this chapter. Almost all countries have 
put in place arrangements that allow for acquisition of full 
or partial qualifications through validation of non-formal 
and informal learning. Several are now working towards 
comprehensive systems offering validation opportuni- 
ties where people live, work and/or study. The develop- 
ment of NOFs, as well as the general acceptance of the 
learning outcome approach, has been an important driv- 


(*) How to make learning visible, 28 and 29 November 2076, Thessaloniki. 
http://www.cedefop.europa.eu/en/events-and-projects/events/ 
how-make-learning-visible 


ANT, (Chri, DIS, ES, |All, le, ISS, WIL, IBM, IMA, INIE, INKO), IIL, SI, 
UK (E&NI) 


er for validation. It can be argued that we have reached 
a ‘tipping point’ in national policy formulation. Validation 
is now an explicit and visible part of lifelong learning and 
(to some extent and importantly) employment and in- 
tegration policies. This ‘tipping point’ has been reached 
through intense interaction between stakeholders at na- 
tional and European level. In this sense validation serves 
as an example of ‘the open method of coordination’ pro- 
moted by the EU during the past few decades. 


It is, however, more difficult to determine whether accept- 
ance and integration at national policy formulation level 
is translated into acceptance and internalisation at practi- 
cal implementation level. Policy documents and plans do 
not necessarily trigger adequate resourcing and financing 
for implementation at local and regional levels. Develop- 
ments during the coming decade(s) will show whether 
acceptance of validation at national policy level will be 
translated into acceptance and internalisation at the level 
of practitioners and among end-users. 
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QUALITY ASSURANCE OF QUALIFICATIONS: BUILDING TRUST 


AND ENHANCING VALUE 


Borhene Chakroun, UNESCO and George Kostakis, Cedefop 


Summary 


This chapter presents recent trends and major challenges 
in developing effective and trustworthy quality assurance 
of qualifications. It analyses why the quality assurance of 
qualifications has become an important issue on policy 
agendas, what the challenges are, and how countries in 
Europe, the Asia-Pacific and elsewhere are responding. It 
identifies issues in assuring the quality and consistency 
of the certification process so that certificates have cur 
rency and are a valid and reliable testament to a learner's 
knowledge, skills and wider competences. The conclu- 
sions reached focus on national and international issues 
facing countries, qualifications systems reforms that are 
underway and further steps that can be taken. 


Keywords: qualification; quality assurance (QA); assess- 
ment; learning outcomes 


6.1. Introduction 


UNESCO's Member States have identified an urgent need 
to reform their qualifications systems. An important driver 
for these reform strategies is to make sure that different 
aspects of national qualifications are useful and consistent 
with one another, and that they are trustworthy and recog- 
nised within and beyond national boundaries. 


At the Third international congress on technical and voca- 
tional education and training (TVET), held in Shanghai in 
2012, participants recognised the importance of quality as- 
surance of qualifications. The Shanghai consensus (') rec- 
ommended exploring the possibility of developing quality 
assurance (OA) guidelines for the recognition of qualifica- 
tions, based on learning outcomes. 


In recent years, TVET has been gathering momentum and 
global attention. Education 2030 devotes considerable 
attention to the development of technical and vocational 


(') See: http:/www.unesco.org/fileadmin/MULTIMEDIA/HO/ED/pdf/ 
concensus-en.pdf 


skills. TVET is also starting to feature in the strategic and 
operational planning of regional economic communities 
(RECs), such as the African Union (AU), the Caribbean 
Community (CARICOM) and the European Union (EU). 
In Africa, for example, the African Union Heads of State 
have adopted the Continental Education Strategy for Africa 
(CESA 2016-25) as the framework for a transformative ed- 
ucation and training system, reinforcing the importance of 
TVET. In 2015, CARICOM adopted a regional TVET strategy 
for workforce development and economic competitive- 
ness with a focus on ‘redefining TVET’ and on ‘developing 
a CARICOM training system underpinned by a quality as- 
surance framework’ (?). 


At European level, countries have been working intensive- 
ly on the comparability of their qualifications, qualifications 
recognition, and periods of learning undertaken in different 
countries to make possible the portability of learner and 
worker qualifications between different EU countries. The 
European qualifications framework (EQF) established in 
2008 — a transnational meta framework based on learning 
outcomes — provided the central point of reference (°) in 
using a translation device of eight European generic levels 
of learning to make qualifications systems more transpar 
ent to employers, learners, qualifications authorities, and 
education and training providers. Quality assurance is a 
key principle of the EQF and is considered the backbone 
for mutual trust between countries and systems in the ref- 
erencing process, as it calls for national quality assurance 
systems to be aligned with the relevant European princi- 
ples and guidelines in vocational training (*) and higher ed- 
ucation (°). In 2015 the ministers in charge of TVET from 


(?) https://dl.dropboxusercontent.com/u/86702287/CARICOM_Car 
ribean_Report_September % 2012.pdf 

(?) EQF referencing is the process that results in the establishment 
of a relationship between national qualifications systems/frame- 
works and their levels to the EOF 

(*) The European quality assurance reference framework for vo- 
cational education and training (EQAVET) is a meta framework 
established in 2009 providing a systematic approach to quality 
assurance at system and provider. http://eurlex.europa.eu/le- 
gal-content/EN/TXT/PDF/?uri= CELEX:32009H0708(01)&from=EN 

(°) See Standards and guidelines for quality assurance in the Euro- 
pean higher education area. http://www.enga.eu/wp-content/up- 
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the European Union Member States, candidate countries, 
Iceland, Liechtenstein and Norway endorsed the new me- 
dium-term deliverables for vocational education and train- 
ing until 2020, known as the Riga conclusions (§). They 
renewed their commitment to continue developing their 
quality assurance mechanisms in TVET and to establish 
feedback loops to inform the renewal and review of their 
TVET qualifications. The European Commission also adopt- 
ed in 2016 its New skills agenda for Europe. This aims to 
ensure that citizens are developing the skills necessary not 
only for today’s jobs but those of tomorrow (7). One of the 
key strands of the Agenda calls for further work at Euro- 
pean level to improve the transparency, comparability and 
trust in qualifications by shifting the focus from duration of 
qualifications to the learning outcomes achieved. 


The recommendations on quality assurance of qualifica- 
tions were further echoed in UNESCO's new Strategy for 
TVET (2016-21) and in the organisation's revised recom- 
mendation concerning TVET (2015). The revised 2017 Coun- 
cil EQF recommendation for lifelong learning (°) also calls 
for transparent and quality-assured TVET qualifications and 
the promotion of mutual recognition of qualifications at na- 
tional, regional and international levels. 


6.2. Recent shifts of focus: from inputs to outputs 


While there is no unique definition of ‘quality assurance 
of TVET’, the most common understanding is that it con- 
sists of a totality of the principles, methodologies, actions, 
measures and instruments through which quality in TVET 
is assured, at system, provider, programme and teaching 
staff level (see Table 6-1 for explanation and examples). 


For many years, the attention of decision-makers has 
been mostly at provider level, often through accredita- 
tion, and on external evaluation conducted by school in- 
spectorates or bodies with similar functions (ETF 2016). 
The focus was to ensure the quality of inputs and pro- 
cesses, such as management of infrastructure, person- 
nel and administrative arrangements as well as the con- 
tent (the curriculum) and delivery of teaching (Cedefop, 
2009). Strategies for monitoring compliance include au- 


loads/2015/11/ESG_2015.pdf 

(°) http://ec.europa.eu/dgs/education_culture/repository/education/ 
policy/vocational-policy/doc/2015-riga-conclusions_en.pdf 

(’) https://ec.europa.eu/transparency/regdoc/rep/1/2016/EN/1-2016- 
381-EN-F1-1.PDF 

(8) http://data.consilium.europa.eu/doc/document/ST-9620-2017-INIT/ 
en/pdf 
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dit/evaluation, continuous monitoring, penalties and the 
publication of outcomes achieved by providers (Bateman 
et al., 2012). In addition to external evaluation/audit and 
inspections, attention was given to self-assessment in 
establishing a quality culture within TVET institutions. For 
example the European reference framework for quality 
assurance in TVET at system and provider level (EQAVET) 
(°) provides a systematic approach to quality assurance, 
promoting a culture of continuous improvement by com- 
bining internal and external evaluation with the use of 
indicators and qualitative analysis. 


While this focus is reasonable and necessary, the continu- 
ing shift to learning outcomes, which allows qualifications 
to be acquired through different learning pathways, sug- 
gests that ensuring the quality of learning provision can- 
not be the only element underpinning the award of quall- 
fications (Cedefop, 2015). Quality assurance of alternative 
pathways — such as work-based learning including appren- 
ticeship, validation of prior learning and online learning — 
can only be ensured by developing reliable ways of decid- 
ing whether the related learning outcomes (what a learner 
knows, understands and is able to do) have been met. 


In recent years, the widespread development and imple- 
mentation of national qualifications frameworks (NOFs) 
has provided a means for reinforcing attention on the qual- 
ity assurance of qualifications. This attention is motivated 
by the efforts of most countries to increase the relevance 
and value of qualifications, for both the individual and the 
labour market. It is also triggered by the growing mobility 
of learners and workers; it needs to provide assurances 
that national qualifications meet regional and international 
standards. As a result, much discussion has taken place 
on the relationship between formal systems of quality as- 
surance and national qualifications frameworks. There is 
broad agreement on connecting the two but opinions differ 
on how this can be operationalised. 


Although quality assurance and qualifications frameworks 
do not share identical origins, they have a lot in common. 
One such common feature is their objective for transpar 
ency and to build mutual trust between national education 
and training systems (Murray, 2013). NOFs help commu- 
nication and transparency by providing the platform for 
different stakeholders to be involved in the development 
and review of standards, a key element that ensures trust 


(°) See EQAVET recommendation 2009: http://eurlex.europa.eu/le- 
gal-content/EN/TXT/PDF/?uriz= CELEX:32009H0708(01)&from=EN 
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Table 6-1. Examples of quality assurance (OA) of TVET 


} DIMENSIONS — MODEL CHARACTERISTICS | EXAMPLES 


System 


Providers 


National 
framework for 
quality that 
consists of an 
overarching set 
of principles, 
actions, 
measures 

and indicators 
covering all parts 
of TVET system. 


Accreditation of 
TVET institutions. 
Accreditation can 
be defined as a 
quality assurance 
process through 
which a training 
provider is 
officially 
recognised 

and approved 

by the relevant 
legislative or 
professional 
authorities 
(Cedefop, 2014). 


Emphasis on building 
a quality culture 
rather than on simply 
complying with 
quality guidelines 
with the active 
participation of all 
actors involved 
(including staff 

and learners) and 

all stakeholders 
(including social 
partners and 
enterprises). 


Focus on the PDCA 
quality cycle ((P)lan, 
(D)o, (C)heck, (A)ct) 
and set of indicators 
covering key building 
blocks including 
inputs-process- 
outputs. 


Encompasses a 
range of standards 
or criteria including 
management, 
infrastructure, 
premises, 
equipment, teacher 
competences, 

as well as those 

of technical and 
administrative 
personnel. 
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In Romania, the national quality assurance 
framework (NOAF) ('°) in TVET has been developed 
from the European common quality assurance 
framework (COAF/EQAVET) in VET. The NOAF 
includes descriptors for seven areas called 
principles including: 


(1) quality management; 

(2) management responsibilities; 

(3) resources management; 

(4) design, development and revision of learning 
programmes; 

(5) teaching, training and learning; 

(6) assessment and certification of learning; 

(7) evaluation and improvement of quality. 


The Tunisian National framework for quality 

of TVET (NFOVET) ("). The structure of the 
NFOVET is strongly based on OA principles, 
encompassing regulations for management of 
TVET institutions (55 standards) and tools for 
measurement (287 indicators). Standards have 
been introduced in vocational training centres as 
part of their recognition. 


New Zealand and Australia use standards and 
indicators to monitor the quality of TVET. The 
standards could apply to the regulator, to providers 
of TVET and to approval of achievement standards 
(competences, qualifications, curriculum). 


Regulations for registration of training providers 
in many countries in different contexts including 
Colombia, Hungary, Ireland, Morocco, Namibia 
and Samoa. 


In Australia ("7), all registered training organisations 
(RTOs) are responsible for ensuring they fully 
comply with the standards at all times as a 
condition of their registration. The purpose of the 
standards is to: 


(a) describe the requirements that an organisation 
must meet to be an RTO in Australia; 

(b) ensure that training delivered by RTOs meets 
industry requirements (as set out in the training 
package or accredited course) and has integrity 
for employment and further study; 

(c) ensure RTOs operate ethically and consider the 
needs of both learners and industry. 


(°) http:/www.erisee.org/downloads/library_romania/framework/Brochure % 200A_EN.pdf 

(") http:/www.etf.europa.eu/pubmgmt.nsf/% 28getAttachment % 29/B42F76F8CCFDFD1AC1257585003470CA/$File/NOTE7QHD7T.pdf 

(2) See the Australian guidelines for Registering Training Organisations (RTOs): https://www.asqa.gov.au/sites/g/files/net2 166/f/Users_Guide_to_ 
the_Standards_for_Registered_Training_Organisations_RTOs_2015.pdf 
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} DIMENSIONS _ MODEL CHARACTERISTICS | EXAMPLES 


Institutions 
self-reviewing 
against a set 

of evaluative 
questions/criteria 
and taking the 
necessary steps 
to improve areas 
of weakness. 


Evaluation and 
review 


Importance of 
self-assessment/ 
evaluation as a 
reflective process in 
which professionals 
take responsibility 
for their own 
assessment and the 
evaluation of their 
organisation. 


Oriented towards 
incorporating 
components 

of international 
standards, such as 
ISO 9000. 


A wide range of countries is encouraging self- 
assessment of training providers as part of the 
accreditation process. A case in point is the New 
Zealand Qualifications Authority (NZOA) ('S) which 
uses rigorous processes and systems, based on 

an evaluative quality assurance framework. This 
includes self-assessment conducted by TVET 
institutions (non-university tertiary education 
organisations (TEOs) in this case). NZOA does not 
prescribe how tertiary organisations do this, as every 
organisation is different, but has published evaluation 
indicators as acommon guide for TEOs and NZOA to 
reach consistent evidence-based judgements. 


TEO self-assessment information provides 
the evidence base for all the quality assurance 
processes. 


Self-assessment across an organisation has four 
main components: 


(a) systematic data gathering; 

(b) robust data analysis that leads to valid 
conclusions; 

(c) reflective processes that involve all people in 
the organisation; 

(d) decision-making for continuing improvement 
connected to the outcomes of a self- 
reflective process. 


The use of ISO 9000 and other types of award 
gained traction in many part of the world. In Latin 
America ('4) this includes Brazil, Chile, Costa Rica 
and Peru. In Brazil, the SENAITECs are poles of 
generation, adaptation and transfer of technology. 
They develop vocational training activities and 
provide services to the industrial sector, such as 
assistance in the productive process, laboratory 
services and technological development and 
information. All this is developed according to the 
strict criteria of the National quality premium and 
the ISO standards. 


('8) http:/Awww.nzqga.govt.nz/providers-partners/qa-system-forteos/self-assessment-overview/ 
(4) https://www.giz.de/expertise/downloads/Fachexpertise/en-tqm-vocational-training.pdf 
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} DIMENSIONS — MODEL CHARACTERISTICS | EXAMPLES 


Programmes Accreditation of Some countries 
programmes distinguish between 

accreditation of 
providers and 
accreditation of 
programmes. The 
latter is an external 
content-review 
process of the 
training programmes 
against a specific set 
of standards. 


It is a process of 
quality assurance 
through which 

a programme 

of education or 
training is officially 
recognised and 
approved by the 
relevant legislative 
or professional 
authorities following 
assessment against 
predetermined 
standards (Cedefop 


2014). 
Teaching staff Teachers In several countries, 
licensing TVET teachers 


including trainers in 
companies, support 
personnel, and 
administrators are 
required to hold a 
licence issued by 

an authority to be 
eligible to teach TVET 
institutions. 


(8) http:/\www.aesl.gov.nl.ca/app/accreditation.html 


This is a model adopted in Canada, Malaysia 
and Samoa. 


In Malaysia all programmes across all TVET 
disciplines would be required to comply with seven 
areas including: 


(a) programme development and delivery; 

(b) assessment of student learning; 

(c) student selection and support services; 

(d) teaching staff; 

(e) educational resources; 

(f) programme management; 

(g) programme monitoring, review and continual 
quality improvement. 


In Canada ('*), apprenticeship programme 
accreditation is a process for assessing education 
programmes against apprenticeship standards 
established by the Provincial Apprenticeship and 
Certification Board (PACB). 


The objectives of apprenticeship programme 
accreditation are to ensure that programmes 
conform to the standards identified in the national 
occupational analyses and reflected in the 
provincial plans of training established for specific 
trades, and that graduates have the required level 
of competence to enter the industry job market. 


The Vocational technical education regulations and 
the Guidelines for vocational technical education 
programmes and educator licensure in the United 
States (for example Massachusetts) ("9). 


In Germany, statutory provisions for the dual 
system specify that, along with professional 
knowledge, skills and competences, trainers must 
also have the skills relating to occupation and work 
education necessary to convey the content of the 
vocational education and training. Personal aptitude 
is also required. Those responsible for vocational 
education and training in companies must be able 
to prove that they are technically and personally 
suited for this task. This usually occurs via an 
examination in accordance with the Ordinance on 
trainer aptitude (AEVO) ("). 


('®) http:/Awww.mass.gov/edu/government/departments-and-boards/ese/programs/educatoreffectiveness/licensure/ 
() https:/Awww.bibb.de/dokumente/pdf/BIBB_VET_Data_Report_2015_zur_BF pdf 
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| DIMENSIONS — MODEL CHARACTERISTICS | EXAMPLES 


Teaching staff Assessors Defines 
licensing responsibilities, 
criteria and 


processes for 
selecting and 


approving assessors. 


Source: Authors’ compilation from different sources. 


in qualifications (Coles, 2016). While quality assurance in 
many countries is a core component of an NOF, in others 
it is seen as a parallel development strongly linked to the 
NOF, as in Australia and New Zealand. This is also the case 
in Indonesia, Malaysia, Philippines and Thailand. In many 
countries quality assurance bodies play an important role 
in implementing qualifications frameworks. In the refer 
encing process of European national qualifications frame- 
works to the EOF the national quality assurance bodies 
must provide their written agreement. Some countries 
(such as Ireland) in their effort to create synergies and 
bring their NOF and quality assurance systems closer, es- 
tablish a single body to maintain and promote their NOF 
and to ensure qualifications quality. 


6.3. TVET qualification quality assurance 
definitions and concepts 


UNESCO seesTVET as ‘comprising education, training and 
skills development relating to a wide range of occupational 
fields, production, services and livelihoods. TVET, as part of 
lifelong learning, can take place at secondary, post-second- 
ary and tertiary levels and includes work-based learning 
and continuing training and professional development that 
may lead to qualifications. TVET also includes a wide range 
of skills development opportunities attuned to national 
and local contexts. Learning to learn, the development of 
literacy and numeracy skills, transversal skills and citizen- 


('8) http:/Awww.cedefop.europa.eu/en/publications-and-resources/ 
publications/5551 


In Europe, responsibility for selecting or approving 
assessors varies between TVET providers (in the 
Netherlands and UK-England), regional school 
inspectorates (in the Austrian school-based 
system, in Romania) or social partners such as 
chambers of commerce, chambers of labour (in 
Denmark and in the dual systems in Germany and 
Austria) ('8). 


Some countries develop national registers for 
members of vocational examination committees; 
examples include Hungary in Europe and Fiji in the 
Pacific. These are the ones that are recognised by 
the regulator to assess ‘national qualifications’ as 
opposed to assessors appointed by the provider. 


ship skills are integral components of TVET’ ("9). Working 
on quality assurance of qualifications refers to the devel- 
opment and use of a range of terms and concepts that 
are essential for understanding and engaging in developing 
effective mechanisms. 


To understand how a qualification can be trusted it is impor 
tant to look at the definition of qualification and identify the 
key elements that need to be quality assured. These are: 


(a) the learning outcomes: the knowledge, skills and com- 
petences that the holder of a qualification should have; 

(b) the assessment and validation process; 

(c) the standards against which learning, assessment and 
validation have taken place. 


The 2017 Council recommendation (7°) on the European 
qualifications framework for lifelong learning acknowl- 
edges the importance of the above elements focusing on 
the quality assurance of learning outcomes, particularly 
qualifications. It refers more specifically to the design of 
qualifications and the application of the learning outcomes 
approach underlying the need to ensuring valid and relia- 
ble assessment according to agreed and transparent learn- 
ing-outcomes-based standards. 


('8) UNESCO (2015). Recommendation concerning TVET. http://por 
tal.unesco.org/en/ev.php-URL_ID=49355&URL_DO=DO_TOP- 
IC&URL_SECTION=201.htm! 

(9) http://data.consilium.europa.eu/doc/document/ST-9620-2017-INIT/ 
en/pdf 
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Table 6-2. Key terms for quality assurance of qualifications (?') 


Learning 


This is the basis for obtaining a qualification. Learning can occur in formal settings or through experiences such 
as work or social activities. Learning can manifest itself through knowledge, skills or wider competences such as 
personal and social competences. 


Assessment 


This is the process of judging an individual's knowledge, skills and wider competences against criteria such as 
learning outcomes or standards of competence. 


Validation 


This is the confirmation that the assessment outcomes of an individual's learning meets predetermined criteria 
(standards) and that a valid assessment procedure was followed. This means that the assessed outcomes have 
been quality assured and can be trusted. Sometimes during this process grading of certain standards of assessed 
outcomes can lead to grades being awarded to candidates. 


Certification 


This is a record of an individual's learning that has been validated. A certificate is usually issued by a body which has 
public trust, displays competence, and confers official recognition of an individual's value in the labour market and in 
further education and training. 


Qualification 
The word ‘qualification’ covers different aspects: 


(a) qualification is seen as the certificate, diploma or title that is awarded by a competent body and which testifies 
that an individual has achieved learning outcomes to given standards. The certificate, diploma or title confers 
official recognition of the value of learning outcomes in the labour market and in education and training. A 
qualification can be a legal entitlement to practise a trade; 

(b) qualification is seen as the knowledge, aptitudes and skills required to perform specific tasks attached to a 
particular work position; 

(c) qualifications express what people know, understand and are able to do and they can take different forms such 
as a diploma or certificate. 


Recognition 
Recognition can be seen in the raised self-esteem of individuals and when it results in their progress into a new job, 
higher pay and/or increased social status. 


Source: Bateman et al. (2016) and Cedefop (2015). 


(*') Adapted from Cedefop (2008, 2015); Eurydice (2006); European Training Foundation (1997); OECD (2007); ILO (1998). 
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Figure 6-1. Quality assurance of the qualifications process 


lification 
Qualifications Assessment 


Qualifications 5 are placed in the Sainetianneere > processes are 5 Qualifications 5 


are developed qualifications 


system accredited 


Source: Adapted from ETF (2016) and Bateman et al. (2016). 


Quality assurance of qualifications varies from setting to 
setting but, in most cases, it includes a chain of processes 
(Figure 6-1) from the development of a qualification to the 
certification of learners during which the above elements 
have been addressed consistently. 


Qualifications are developed: institutions or stakeholders 
decide to develop a new qualification or to revise an old 
one; for example a TVET private provider, an awarding 
body, or sectoral branch. 


Qualifications are placed in the qualifications system: to be 
entered into the NOF qualifications will have to meet cri- 
teria, which ensure their quality. New, revised or amended 
qualifications must be validated and approved to ensure 
that they have clear and relevant purposes and have been 
designed to meet these purposes; for example, that there 
is a demonstrable need for the qualification and that the 
qualification has been well designed to meet this need. 


Assessment institutions are accredited: to assess learners 
for qualifications in the NOF and to issue certificates to 
successful learners, institutions will have to meet certain 
criteria, ensuring their capacity to carry out these functions 
to an appropriate level of quality. For example, the institu- 
tion must have an effective approach to assessment and 
the quality management of assessment, the necessary 
material resources to assess the qualifications/units, and 
the necessary qualified staff to assess and certificate the 
qualifications/units. Assessment should be valid and fair, 
reliable and transparent. 


Qualification processes are quality assured: external 
quality assurance of assessment and certification is re- 
quired. This can be checked by external verification and 
audit of assessment institutions run by concerned reg- 
ulatory bodies. 


Qualifications are endorsed: certificates for qualifications 
in the NOF will be endorsed by the regulatory body (for ex- 
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Successful 
learners are 
awarded 
qualifications 


Qualification 


quality assured are endorsed 


ample the qualification authority) to show that they meet 
all of the quality requirements. Endorsement will be a sig- 
nal that the qualification in question is in the NOF and that 
award of the qualification has been made by an approved 
institution on the basis of quality assured assessment. For 
example, the regulatory body (such as the national qualli- 
fication authority) could issue supplies of endorsed paper 
for the use by the awarding institutions in issuing certif- 
icates or could issue an additional document (a covering 
document) to accompany the certificate issued by the 
awarding institution. 


Successful learners are awarded a certificate: certification 
is the process of recognising the achievements of candi- 
dates. Qualifications are often issued by the same body 
that has conducted or organised the final assessment, 
such as by schools/VET providers on behalf of the compe- 
tent national authority (in the German and Austrian school- 
based schemes, the Netherlands, Portugal and Slovenia), 
by regional chambers (in the German and Austrian dual 
systems) or by both (Denmark) (Cedefop, 2015). 


The different institutions involved in this process must 
act to ensure continuing quality improvement in all 
these areas. 


6.4. Quality assurance trends in qualifications 


Faced with a myriad of challenges including employability 
and unemployment, particularly though not exclusively for 
the young, countries are seeking to reform their qualifica- 
tions to ensure underpinning by quality assurance process- 
es. Such processes are a key part of the solution to mak- 
ing TVET qualifications more trustworthy and more highly 
valued in the labour market. However, for most countries, 
robust quality assurance of qualification systems remains 
an objective to be realised. 


In 2016, UNESCO conducted a review of TVET quality assur 
ance mechanisms qualifications in Asia-Pacific (Bateman 
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and Coles, forthcoming). The key conclusion of the review 
was that developing quality assurance in TVET is more 
than just a technical process. Political, economic, social, 
cultural and financial considerations, as well as institution- 
al structures and legal frameworks, shape the criteria and 
processes that underpin TVET quality assurance (Bateman 
and Coles, forthcoming, p. 11). While recognising uneven 
progress among countries, the review suggested several 
areas for improvement, formulating 13 recommendations 
as the basis for forthcoming regional guidelines on quality 
assurance of qualifications: 


(a) the role of governance: countries should conduct a stra- 

tegic review on the structure of governance of quality 

assurance across TVET with a focus on strengthening 
coordination and accountability of all aspects and bodies 
responsible for the qualification process; 

links between quality assurance and qualifications frame- 

works: qualifications frameworks need to have a quality 

assurance dimension and/or be linked to quality assurance 
frameworks, not only at national level but also regional; 
improving stakeholder engagement: reflect on the level 
of engagement of industry, employers and civil society in 
the quality assurance of TVET qualifications, and design 
the system to maximise engagement; 

increasing transparency: consider increasing transparen- 

cy of processes and publishing outcomes of quality as- 

suring activities and research to broaden understanding 
in the wider community; 

(e) developing a quality culture: develop a process of self-as- 
sessment and continuous improvement in TVET, making 
sure that quality assurance is fit for purpose; 

(f) ensuring funding and resources: targeted funding is crit- 

ical to the implementation of a system for quality assur 

ing TVET qualifications; 

increasing the use of learning outcomes approaches: de- 

veloping greater use of learning outcomes should be a 

priority; current initiatives on greater use of learning out- 

comes to make assessment of learning more objective 
should be supported; 

attention to validation and certification: 

should ensure that when designing or reviewing the 

system for quality assuring TVET qualifications, atten- 
tion is paid to the validation and certification aspects of 
the qualification process; 

(i) building capacity of assessors: promote the capacity 
development of assessors, which could be achieved 
through constructing an assessor qualification, and train 
assessors in assessment; 


(b 


Sc 


(d 


(g 


= 


countries 
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(i) access to and pathways of TVET qualifications: under 

take research into TVET qualifications to explore how 

they support access for all (especially vulnerable groups) 
through different pathways, recognition of prior learning 
and credit transfer; 

improving development and use of data systems: make 

the development and use of data systems, including la- 

bour market information systems, an integral part of the 
whole process of quality assurance of TVET qualifications; 

(I) Supporting regional cooperation and policy learning: there 
is much to gain for all countries in participating in further 
collaboration, and the sharing of experience. A communi- 
ty of practice should be created which has the potential 
to inform policy and practice. A regional collaboration plat- 
form could be an instrument to support policy learning, 
as is the opportunity to take advantage of the range of 
cross-national TVET networks already available. Regional 
guidelines for the quality assurance of TVET qualifications 
could be an approach for participating countries; 

(m) developing guidelines and support materials: it is pro- 
posed that guidelines be developed to strengthen com- 
mon understanding of quality assurance systems and 
provide a basis for enhancement in quality assurance 
systems in the Asia Pacific. 


(k 


The findings of a recent Cedefop study (Cedefop, 2015) 
on quality assurance of certification in the European con- 
text, echo those of UNESCO in Asia-Pacific. The study 
calls for reliability and validity in the qualification process 
in technical and vocational education and training (TVET) 
and identifies eight related quality features. It concludes 
that ‘to strengthen trust in certification, results across the 
system based on the same qualification standards must be 
comparable. Comparability of results ensures that holders 
of the same qualification have actually achieved the learn- 
ing outcomes required for it and therefore qualifications 
can be trusted’ (Cedefop, 2015). It is important that same 
assessment results can be obtained in different cases (in 
relation to context, time, assessors or assessment tasks) 
and that assessment outcomes that lead to the award of 
a qualification are consistent. The study also calls for the 
involvement of labour market stakeholders in developing 
assessment standards based on learning outcomes and 
for their participation in summative assessment. Their 
participation ensures the validity of assessment and that 
assessment tasks measure as precisely as possible the 
learning outcomes expected of the holder of a qualifica- 
tion. Their involvement may also increase credibility and 
trust in qualifications. 


GLOBAL INVENTORY OF NATIONAL AND 
REGIONAL QUALIFICATIONS FRAMEWORKS 


In another context, the European Training Foundation (ETF) 
conducted a review of quality assurance mechanisms in its 
partner countries, which include those in the Arab region, 
Balkans, Eastern Europe, Caucasus and Central Asia. In 
these countries, there seems to be new emphasis on the 
processes of developing the standards behind a qualifica- 
tion, in terms of how and by whom it is done. In its recent 
toolkit on developing better qualification systems, the ETF 
focuses on the issue of quality assurance, providing a set 
of recommendations for countries that wish to strengthen 
the OA of their qualifications (ETF 2016). ETF work em- 
phasises the importance of quality assuring the standards 
behind qualifications, the assessment processes, and cer 
tification, and includes an independent validation check at 
every step (ETF 2016, p. 121). 


6.5. Regional quality assurance dimensions 


The application of quality assurance systems to transna- 
tional TVET is becoming increasingly important, with quali- 
ty assurance of qualifications addressed in different ways. 
Alongside regional qualifications frameworks there are in- 
creasingly coordinated approaches/frameworks of quality 
assurance, such as the East Asia summit (EAS) TVET qual- 
ity assurance framework, the European quality assurance 
in vocational education and training and the Pacific quality 
assurance framework. The key aims of a regional quality 
assurance framework are to develop mutual understanding 
and trust among member countries. Some regional quali- 
fications frameworks, such as the European qualifications 
framework (EQF) and Association of Southeast Asian Na- 
tions (ASEAN) qualifications reference framework, require 
countries to refer to agreed quality assurance frameworks 
in the referencing process. 


6.6. Conclusion 


Quality assurance of TVET qualifications is evolving in 
many countries. Often this evolution is related to other 
reforms in TVET systems: development of national quali- 
fications frameworks and moves towards a learning out- 
comes approach; reforms of governance and involvement 
of social partners and other stakeholders; shifts towards 
more autonomy of TVET providers; expanding and diversi- 
fying access and increasing attention to work-based learn- 
ing (UNESCO, 2015; Cedefop, 2016; Bateman et al., 2016). 
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Given the diversity of country contexts, much consensus 
exists around the idea that quality assurance is a crucial ve- 
hicle for trust and value of qualifications. Giving attention 
to the different elements of the quality assurance chain is 
an appropriate approach to assessing the relevance, relia- 
bility and validity of the processes in place. 


Although quality assurance has risen to the top of policy 
agendas, the capacity of TVET systems to respond to the 
requirements of the quality assurance chain is often lim- 
ited. Countries should conduct a strategic review on the 
governance structure of quality assurance across TVET, 
with a focus on strengthening coordination, capacities and 
accountability of all aspects of the qualification process 
(Bateman and Coles, forthcoming). 


Despite growing international interest, there is little 
empirical research into the impact of introducing quali- 
ty assurance of qualifications on the value and currency 
of TVET qualifications in the labour market within and 
across countries. 
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Table 6-3. Key regional initiatives on quality assurance 


INITIATIVE PURPOSE PROGRESS 


European quality 
assurance reference 
framework for VET 


In Europe, 

the 2017 EOF 
recommendation 
for lifelong learning 
lays out in Annex 
Ill the principles of 
quality assurance 
that must underpin 
implementation of 
the framework 


East Asia summit 
TVET quality 
assurance 
framework 


Pacific quality 
assurance 
framework 


(22 
(23 
(24 


(28 


The key purpose of the European quality assurance 
reference framework for TVET is to serve as a 
reference instrument to promote and monitor 
continuous improvement of TVET systems (77). The 
framework is based on the continuous improvement 
cycle (73) and includes: quality criteria; indicative 
descriptors for TVET system levels; indicative 
descriptors for TVET provider levels; and a reference 
set of quality indicators for assessing quality in 
TVET. The indicators proposed cover relevance; 
investment in teachers; participation, completion 
and placement rates; utilisation of acquired skills 

at the workplace; unemployment rate; prevalence 
of vulnerable groups; mechanism to identify 
training needs and schemes used to promote 
better access to TVET. The EQAVET framework 
does not specifically cover the quality assurance of 
qualification design, assessment and certification 


10 criteria are governing referencing of national 
qualifications frameworks to the EOF Criteria 5 
and 6 explicitly refer to the need to document 
existing quality assurance arrangements and their 
consistency with the relevant European principles 
and guidelines, underlining the importance of these 
for ensuring confidence when comparing national 
qualifications frameworks. 


The East Asia summit TVET quality assurance 
framework aims to help EAS countries to benchmark 
their TVET quality assurance systems to an agreed 
international framework. It includes standards for 
competent bodies and for TVET providers. 


The quality assurance framework includes standards 
based benchmarks for national competent bodies 
and for providers of education and training services 
across the Pacific. The Pacific nations use these 
quality assurance standards to benchmark their own 
quality assurance standards and processes. In a few 
instances, Pacific nations have adopted the regional 
qualifications framework and the quality assurance 
framework at the national level. 


Official Journal of the European Union, C 155, 8.72009, pp. 1-10. 
Plan, implement, evaluate and review. 

http:/www.eqgavet.eu/gns/policy-context/european-quality-assurance-reference-framework/framework-implementation.aspx 
https://ec.europa.eu/ploteus/documentation#documentation_73 


All EU-28 countries have 
established OA approaches 
compatible with EQAVET (7). In 
34% of the countries, the EQAVET 
framework was used to develop 
the national OA approach. 


By April 2016, 32 countries had 
linked (‘referenced’) their national 
qualifications levels to the EQF (”). 


Currently, nine East Asia summit 
countries have conducted self- 
assessment against the competent 
body standards (Cambodia, 

India, Indonesia, Laos, Malaysia, 
Myanmar, Philippines, Thailand and 
Vietnam). 


Pacific quality assurance 
framework was adopted in 2015. 
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INITIATIVE PURPOSE PROGRESS 


South African Quality assurance guidelines were developed To ensure that credible, 

quality assurance to support the implementation of the Southern trustworthy information is being 

framework African Development Community (SADC) regional shared across SADC, a regional 
qualifications framework. qualifications verification network 


(SADCOVN) was established. 
The SADCOVN was initiated to 
strengthen verification in SADC 
and has applied for membership 
of the African qualifications 
verification network to ensure 
that African qualifications can be 
trusted. Currently, the SADCOVN 
document is being translated and 
prepared for publication. 


The SADCOF encourages SADC countries to have 
good OA mechanisms. In this regard, the SADCOF 
links up with regional OA bodies. 


There are plans to create a list of 
credible SADC institutions and 
their qualifications. 


NB: EQAVET: quality assurance in vocational education and training; EQF: European qualifications framework; OA: quality assurance; SADC: 
Southern African Development Community; SADCOF: SADC qualifications framework; SADCOVN: SADC qualifications verification network. 


Source: Author, based on different sources. 
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CHAPTER 7. 


he) 


LEARNING OUTCOMES IN COMPARING VET QUALIFICATION 
PROFILES: A GLOBAL APPROACH 


Jens Bjgrnavold, Cedefop, and Borhene Chakroun, UNESCO 


Summary 


This chapter summarises the outcomes of joint research 
carried out by Cedefop, the ETF and UNESCO in 2015-16 
comparing the content and profile of four vocational qual- 
ifications (bricklayer, healthcare assistant, hotel reception- 
ist and ICT service technician) in 26 countries worldwide. 
The research was made possible by the increasing use of 
learning outcomes for defining and describing qualifica- 
tions. Comparison was also made possible by the contin- 
uing development of the European classification of skills, 
competences, occupations and qualifications (ESCO) pro- 
viding a reference point for comparison of national profiles. 
We argue that systematically demonstrating similarities 
and differences between qualifications allows national 
stakeholders to reflect better on their own choices and 
priorities: are, for example, significant differences in skills 
requirements the result of differing national settings and 
requirements, or are they merely caused by lack of infor 
mation and oversight? 


Keywords: technical and vocational education and training 
(TVET) qualifications; learning outcomes; European classi- 
fication of skills, competences, occupations and qualifica- 
tions (ESCO) 


7.1. Introduction 


While qualifications frameworks are developed for many 
different purposes, the promotion of learning outcomes 
stands out as probably the most important. Systemat- 
ic use of learning outcomes in frameworks and qualifi- 
cations allows for the introduction of levels supporting 
comparison and transfer of qualifications; outcomes in- 
crease the overall transparency of qualifications, making 
it possible for individual learners and future employers 
to judge their quality and relevance. The approach also 
points towards a ‘common language’ allowing dialogue 
between education and training institutions and external 
stakeholders on skills needs and relevant responses. Dif- 
ferent studies (Cedefop, 2016; Keevy, Chakroun (2015), 
show that national qualifications frameworks have played 


a significant role in supporting more systematic use of 
learning outcomes in many countries. Compared to the 
situation in 2013 when the first global inventory was pro- 
duced (ETF, Cedefop and UNESCO-UIL, 2013), this in- 
ventory indicates that learning outcomes are now more 
widely used and considered by countries in developing, 
reviewing and updating their qualifications. 


This chapter draws on the increased use of learning 
outcomes, supported by qualifications frameworks, and 
uses these statements as a basis for comparing the con- 
tent and profile of vocational qualifications in 26 coun- 
tries worldwide ('). 


7.2. The comparative study: focus and methodology 


In 2016, Cedefop, ETF (?) and UNESCO, in cooperation with 
national stakeholders (°), carried out a comparative study 
of vocational education and training qualifications for four 
broadly defined profiles in 26 countries across the world. The 
study was inspired by the fact that an increasing number 
of countries now use learning-outcomes-based approaches 
to define and write qualifications (4). The emergence of this 
‘common language’ (°) makes it possible to compare what 


(') The study was carried out by researchers from 3s in Austria and 
Ockham IPS in the Netherlands, Karin Luomi Messerer, Monika 
Auzinger and Simon Broek. The findings presented in this chapter 
are based on the draft reports to Cedefop and UNESCO. 

(2) The European Training Foundation, represented by Eduarda Castel 
Branco, added data for VET qualifications in Albania, the Former 
Yugoslav Republic of Macedonia and Tunisia. 

(?) The analysis of national level data was supported by several indi- 
vidual experts and institutions. 

(*) The global study was inspired by, and based on, a European study 
developed by Cedefop (2017), looking at 10 VET qualifications in 
10 EU countries. These 10 qualifications are: bricklayer/masonry, 
health care assistant, hotel assistant/receptionist, ICT service 
technician, plumber (cooling and heating), sales assistant, den- 
tal assistant, logistics technician, machine operator (automation/ 
CNC) and farm management professional (agriculture). 

(() See for example Cedefop (2016). The application of learning 
outcomes approaches across Europe: a comparative study. Lux- 
embourg: Publication Office. Cedefop reference series; No 105. 
http://www.cedefop.europa.eu/en/publications-and-resources/ 
publications/3074 


96 


Table 7-1. Qualifications covered by the study 
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GENERIC TITLE FURTHER INFORMATION/FOCAL POINT OF QUALIFICATIONS SELECTED 


Bricklayer/Masonry 


Bricklayer is a traditional qualification in the construction sector (house building, 


commercial building, restoration, repair and maintenance). 


Healthcare assistant 


Healthcare assistants (or assistant nurses) provide assistance, support and direct personal 


care to patients and residents in a variety of institutional settings such as hospitals, 

clinics, nursing homes and aged care facilities. They generally work in support or under the 
guidance of qualified healthcare professionals (often nurses) or associate professionals. 
Might be a relatively new qualification in some countries. 


Hotel assistant/ 
receptionist 


Deals with communications as part of the reception function (e.g. provides tourist 
information to guests), with check-in and check-out of customer as well as with bookings; 


supports administration, book keeping, cost accounting. 


ICT service 
technician 


Source: Cedefop, (2017). 


countries expect their TVET (°) candidates to know, be able 
to do and understand after having successfully completed a 
TVET programme. The study focuses on intended (’) learn- 
ing outcomes and does not claim to map and compare the 
actual achievements of learners, as these are demonstrated 
through assessments or at work (°). While this can be seen 
as limiting the usefulness of the research, the systematic 
insight into national priorities can potentially add significant 
value. The study did not aim for a ranking of countries but to 
examine the transparency and comparability of qualifications 
at international level and to support peer learning. System- 
atically demonstrating similarities and differences between 
qualifications allows national stakeholders to reflect on their 
own choices and priorities: are, for example, significant dif- 
ferences in skills requirements the result of differing national 
settings and requirements or are they merely caused by lack 


(°) For the purposes of this chapter, the term technical and vocational 
education and training (TVET) is used. Cedefop uses the term vo- 
cational education and training (VET). 

(’). The distinction between intended and actually achieved learning 
outcomes is critical for understanding the strengths and the lim- 

itations of the study: intended learning outcomes are those that 

are presented in written documents. They describe what learners 
should know and be able to demonstrate at the end of a learn- 
ing process. They are thus statements of expectations related to 
earners’ performances. Achieved learning outcomes are those 

hat are demonstrated by an individual learner at the end of a 

earning process and/or in a working context. In some countries 

he term ‘competence’ is used to describe these actually achieved 
earning outcomes. 

(°) The European part of the study, carried out by Cedefop, takes one 
step in this direction by analysing how learning outcomes are re- 
viewed and how the experiences from ‘real life’ support continu- 
ous renewal (through a feedback loop) of qualifications. 


Provides ICT support and systems service in companies/institutions; focus is on more 
technical aspects of ICT installation, service and maintenance. 


of information and oversight? The following sections present 
the focus and the methodology of the comparative study. 
Given that this is a pilot-study, testing what is in many ways 
a new approach, this will be presented in some detail. 


7.2.1. The qualifications studied 

The study focuses on qualifications for four profiles com- 
monly awarded in most countries in the world and seeks 
to describe and compare their scope, profile and content. 
The qualifications chosen are presented in Table 7-1. 


The qualifications were selected to illustrate the span of 
knowledge, skills and competences covered by national 
qualifications. While the bricklayer qualification must bal 
ance century-old handicraft traditions with modern indus- 
trial construction, the healthcare assistant and the hotel 
receptionist qualifications need to be able to offer the rel- 
evant personal services, a combination of sectorspecific 
technical knowledge and interpersonal and transversal 
skills and competences. The ICT service technician qualifi- 
cation needs to combine technical and transversals skills 
and competences in a context characterised by rapid tech- 
nological change. At least two (hotel receptionist and ICT 
service technician) of the four qualifications are likely to 
refer to common sets of skills across countries. 


7.2.2. The countries and regions covered 

Starting from a Cedefop project (Cedefop, 2017) com- 
paring 10 TVET qualifications in 10 European Union coun- 
tries, the key purpose of the global study was to extend 
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the selection of countries to cover as wide a variety of 
national systems and solutions as possible. The exten- 
sion of the study from 10 to 26 countries was made pos- 
sible by the cooperation between regional qualifications 
frameworks initiated by UNESCO from 2013 and onwards 
(9). This cooperation was essential for getting access to 
and analysing the relevant data from national qualification 
authorities. While important regions and countries are 
missing, the study covers a wide range of countries and 
regions representing very different national qualifications 
systems. Countries are united, however, by having intro- 
duced learning-outcomes-based national qualifications 
frameworks. The participating countries were divided 
between Africa (five), Asia (two), Europe (12), the Gulf 
region (one), Latin America and Caribbean region (three) 
and the Pacific region (two). Table 7-2 shows the partic- 
ipating countries and the qualifications covered in each 
country and as a total. 


The four selected qualifications are not covered in all 26 
countries. This is partly due to lack of national capacity to 
provide data and partly where a particular qualification is 
not awarded by the national system. This latter is particu- 
larly the case for the healthcare assistant. 


7.2.3. Contextualising the qualifications 

For a comparison of the content and profile of the qualifi- 
cations to be possible, the institutional context in which 
the qualifications operate had to be clarified. While most of 
the 26 countries have set up national qualifications frame- 
works operating with learning-outcomes-based levels, 
only the European countries, through the EOF, have adopt- 
ed a common reference point allowing for comparison of 
these levels. This lack of a common reference provides a 
challenge for the global comparison and could potentially 
be addressed through the gradual adoption of world refer 
ence levels, as currently sought by UNESCO. The occupa- 
tional purpose of each qualification also provides an impor 
tant indication of the role assigned to each qualification. 
The study distinguishes between two main occupational 
roles; skilled and semi-skilled worker. Table 7-3 shows how 
these two roles have been defined. 


(9) In a follow-up to the conclusions of the 2012 Shanghai declara- 
tion on TVET, a working group has been set up to address the 
feasibility of World reference levels for qualifications. Including 
representatives of existing regional qualifications frameworks, 
this working group has initiated joint work in several areas, nota- 
bly seeking to share experiences in the use of learning outcomes 
for different purposes. 
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These roles can be further clarified by identifying the re- 
lationship of these qualifications to jobs and occupations. 
Does the qualification, for example, grant exclusive access 
to an occupation or is it one of several pathways leading 
in this direction? Table 7-4 shows different ways in which 
qualifications can be related to occupations. 


7.2.4. The learning outcomes data 

While the contextual data help to locate the qualifications 
in relation to the national qualifications systems and labour 
markets, analysis of the intended learning outcomes is at 
the centre of the study. The learning outcomes approach 
now is commonly used for describing TVET qualifications 
in the countries covered by the study ("°), but the way it 
is applied and interpreted varies. This means that learning 
outcomes differ in length, level of detail as well as orienta- 
tion. Countries further use different documents and instru- 
ments to define and describe the intended learning out- 
comes and it will often be necessary to combine different 
documents to get full understanding of the qualification in 
question. Countries normally use the following documents 
(Figure 7-1), or combinations of these, when defining the 
content and profile of their TVET qualifications. 


While occupational standards are oriented to meet needs 
of the labour market, specifying the performance require- 
ments in an occupation, educational standards, curricula, 
assessment criteria and programme descriptions focus on 
the learning process and how this can be managed. These 
documents will normally vary significantly in level of detail 
and specificity. While a national qualification standard will 
normally provide an overarching description of the content 
and profile of a qualification, assessment criteria (for exam- 
ple) need to specify not only what is to be assessed but 
also indicate the expected level of performance. 


7.2.5. Reference point for comparing learning-outcomes-based 
qualifications 

Given that qualifications are presented and described in dif- 
ferent ways across countries, identification of similarities 
requires some form of reference point. Experiences from 
European pilot projects point to the need for independent 
reference points (grids or matrices) making it possible to 


(9) The fact that the 26 countries all use learning outcomes to define 
and describe their qualifications signals general acceptance of this 
approach. Differences in interpretation and implementation be- 
tween countries, however, means that the objective of creating ‘a 
common language’ to be used across institutional and geographic 
borders has yet to be fully realised. 
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Table 7-2. Countries and regions covered by the study 


COUNTRIES QUALIFICATION PROFILES COVERED 


BRICKLAYER HOTEL HEALTHCARE | ICT SERVICE 
ASSISTANT/ ASSISTANT TECHNICIAN 
RECEPTIONIST 
Africa Mauritius e @ ® cy 4 
Namibia @ 1 
Tunisia CY @ 2 
South Africa e e e e 4 
Zambia e e o 3 
Asia Korea (KR) e e @ @ 4 
Philippines (PH) e o) e e 4 
Caribbean Barbados e C) e @ 4 
Region 
and Latin Chile e e@ 2. 
America ; 
Costa Rica e ° e e 4 
Europe Albania e e 2 
Austria ® e @ e 4 
Bulgaria e e @ @ 4 
Denmark e) @ oO @ 4 
Finland @ t e o 4 
France e @ @ @ 4 
Ireland @ ° @ 3 
Lithuania e e e @ 4 
Macedonia e 1 
Netherlands @ e e @ 4 
Spain e e e C) 4 
UK-England e e e 8 4 
Middle East United Arab 1 
Emirates bed 
(UEA) 
Pacific New Zealand e ) e © 4 
Region 


Samoa e 


1 
ae [ea ee ee 


Source: Auzinger, M.; Broek, S.; Luomi-Messerer, K., 2017 (forthcoming). 
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Table 7-3. Main roles of TVET qualifications in the labour market 


[ SKILLED WORKER SEMI-SKILLED WORKER 


The qualification attests that the person is 

a qualified/competent/skilled worker, with the 
knowledge, skills and competence required 
to practise an occupation. 


Scope 


Source: Adapted from Spottl and Ruth (2011), pp. 148-151. 


identify similarities and differences. The VOTS model (") 
and projects building on this approach ('?) exemplify this. 
Several ECVET projects have used a similar approach for 
comparing qualifications ('%). These project approaches, 
however, are too limited in scope and left the project with 
two main alternatives: 


(a) the terminology currently being developed in the context 
of the ESCO project (the multilingual classification of Eu- 
ropean skills, competences, qualifications and occupa- 
tions) ("4): 

(b) the terminology developed in the context of O*NET, the 
Occupational information network used in the US ('5). 


It was decided to use a draft version of the ESCO termi- 
nology made available at the beginning of 2016 ('%). This 
reflects that ESCO is explicitly designed to work at inter 
national level and aims to be relevant across national la- 
bour markets and education and training systems. Second, 
ESCO not only provides terminology on occupational tasks 
and functions, but also introduces terminology on knowl- 
edge, skills and competence designed to bridge education 
and work. For the purpose of the study, two ESCO ele- 
ments were used: 


(a) draft ESCO terminology on occupational specific skills 
and competences is developed for 27 sectors, covers 
3 000 occupations and contains approximately 13 000 


(") Luomi-Messerer (2009). http://www.vocationalqualification.net/ 
(2) www.VocationalQualification.net (Resources: Links: VOTS model). 
(8) http:/www.ecvet-projects.eu/ToolBox/Too!lBoxList.aspx?id=28&- 
ype=1 

(4) https://ec.europa.eu/esco/portal/home 

(8) http:/www.onetonline.org/ 

t needs to be emphasised that these are draft results and ESCO 
has been further developed since then and that a first official ver 
sion of ESCO was published in July 2017 Some have pointed out 
he room and need for improvement. 


(‘6 


The qualification attests that a person has obtained 
a set of skills/competences, with the skills needed 

to perform particular tasks/a narrow set of activities 
(semi-skilled worker). 


skill and competence terms ("”). Using ISCO 2011 ('8) asa 
tool for distinguishing sectors and occupations, the skills 
and competence terminology ('°) evolved from analysis 
of main tasks and functions in each occupation and as- 
sociated jobs. This analysis was then adjusted using in- 
formation on learning outcomes for describing relevant 
qualifications (#°). The learning outcomes statements 
identified at national level were mapped to these occu- 
pationally specific lists of terms, demonstrating corre- 
spondence or lack of correspondence; 

(b) draft ESCO terminology on cross-sectoral and trans- 
versal skills and competences is organised according 
to five key headings divided into 16 subheadings: ap- 
plication of knowledge (numeracy/mathematics, ICT, 
Health and safety); language skills and competenc- 
es (including mother tongue and foreign languages); 
thinking skills and competences (problem solving; 
innovative and creative thinking; entrepreneurial think- 
ing; social skills and competences (leading and manag- 
ing others; working with others; training and support- 
ing); critical thinking; learning); and personal attributes 
(physical abilities; attitudes to work; values at work). 
While the fifth headline (attributes) is only partly about 
skills and competences, it is an important category 
for defining learning outcomes. As with the occupa- 
tion-specific skills and competences, national terms 
were mapped to the ESCO list, demonstrating corre- 
spondence or lack of correspondence. 


(') The terminology will be available in 24 languages. 

('8) International standard classification for occupations, 2011 

('®) https://ec.europa.eu/esco/portal/escopedia/Learning_outcome 
(2°) The number of knowledge, skills and competence terms included 
for each occupation varies. For the occupational areas touched by 
this study, the variation is: the profile for bricklayer includes 48 
terms; the hotel assistant/receptionist comprises 26; the profile 
for healthcare assistants is 31 and the ICT service technician in- 
cludes 34. 
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Table 7-4. The relationship between qualifications and occupations 


Licensing refers to situations in which it is unlawful to carry out a specified range of activities for pay, i.e. an occupation 
or profession, without first having obtained a qualification which ensures that the practitioner meets the prescribed 
standards of competence. 


Certification refers to situations in which there are no restrictions on the right to practise an occupation, but job 
holders may voluntarily apply to be certified as competent by a State-appointed regulatory body. 


Registration refers to situations in which it is unlawful to practise without having first registered one’s name and 
address with the appropriate regulatory body. Registration provides some form of legal barrier to entry, but an explicit 
skill standard is not provided. 


Accreditation refers to situations in which an individual may apply to be accredited as competent by a recognised 
professional body or industry association. Accreditation is distinct from certification in that the criteria governing 
accreditation and the procedures regarding enforcement are entirely the responsibility of the accrediting body rather 


than the State (?'). 


Source: Cedefop (2013b). 


Figure 7-1. Documents providing information on learning outcomes 


Occupational standards... may specify ‘the main jobs that people do’, describing the professional tasks and activities  Orentation 
as well as the competences typical of an occupation. Occupational standards answer the question ‘What does the on informing 
student need to be able to do in employment?” on the job 
practice 
V a 
Educational standards... may define the expected outcomes of the learning process, leading to the award of a \ 
qualification, the study programme in terms of content, learning objectives and timetable, as well as teaching methods ! 
and learning settings, such as in-company or school-based learning. Educational standards answer question ‘What does t 
the student need to learn to be effective in employment? 
v v ! 
. . . . 1 
A curriculum is a normative document (or a collection Assessment standards... may specify the object 
of documents) setting the framework for planning tie ' 
learning experiences. Depending on the country, the type SH GESESEHL I) petiomane BRN T a assess TENE 
of education and training, and the institution curricula <> mealies anu pes tn pine yeas 
may define, among other learning outcomes ‘objectives EEE EI aie ceeesoulell Semele 1 
contents place and duration of learning teaching and ELISE) Wale lle por as KW wiliat tie 
: f student has learned and is able to do in employment?’ ! 
assessment methods to a greater or to a lesser extent. ay 
1 Orientation on 
The learning programme is a written document planning learning experiences in a specific learning setting. It is informing mn 
developed on the basis of the curriculum and takes into account the learners’ needs. ee 


Source: Cedefop, 2017. 


(*') Cedefop (2013a). The role of qualifications in governing occupations and professions. Luxembourg: Publications Office. 
Cedefop working paper; No 20. 
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Figure 7-2. Date of issue of documents used 


Number of documents 
pe) BO Go: (se) & 
on { cee J 


0 
15 14 
10 
5 4 
1 
0 
1995-1999 2000-2004 2005-2009 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 


ESCO is applied as a sort of ‘fixed’ terminological 
point allowing for the analysis of national learning-out- 
comes-based data. The fact that the terminology is 
detailed (for example containing 48 terms for captur- 
ing occupational specific skills and competences for 
bricklayers), increases its relevance and allows for 
more precise analysis of the content and profile of 
national qualifications. The combination of occupa- 
tion-specific, cross-sectoral and transversal skills and 
competences was, from the start, considered to be 
crucial to the comparison. 


7.3. Main study findings 


The findings of the study can be divided into two main 
parts: those related to the positioning of qualifications 
in the national education and training and labour mar 
ket context; and those related to the learning outcomes 
and the content and profile of the selected qualifica- 
tions. The following two sections provide an overview 
of main findings. 


7.3.1. Qualifications and context 

It is commonly asserted that TVET qualifications, to 
be relevant to the labour market and society at large, 
must be continuously reviewed and renewed. The study 
shows that most qualifications studied (60 out of 80) had 
been issued during the past six years; 19 are registered 
as having been issued before 2010, raising questions 
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35 


25 


1 


2010-2014 2015-2016 Not yet 


registered 


over their relevance to the labour market they are sup- 
posed to serve (7). 


The level of a qualification can be established either us- 
ing a learning-outcomes-based approach (as provided 
by regional qualifications frameworks such as the EQF) 
or by using institutional levels (such as the international 
standard classification of education, ISCED 2001). These 
approaches are based on different data but provide, 
when combined, a useful picture of the way qualifica- 
tions are levelled. Table 7-5 offers an overview of the 
EOF level and ISCED 2011 classification as indicated 
in the qualifications descriptions. For countries outside 
Europe the EOF level indication can only be estimated, 
as no formal referencing of their NOF levels have taken 
place. In cases where no NOF level is available, an es- 
timation is not possible. A reference to ISCED 2011 is 
indicated for 56 of 80 qualifications (mainly in Europe). 
This can be explained partly by the structure of educa- 
tion and training systems of the concerned countries 
where non-formal TVET is important and lies outside the 
education system. 


While incomplete, the table is still interesting in demon- 
strating the wide range of levels covered by TVET qualifica- 


(72) The question of review and renewal is addressed in the second 
part of the Cedefop study (for 10 countries) and an extension of 
this focus to the remaining 16 countries is being considered. 
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tions, in this case from EOF levels 2 to 5. Table 7-6 shows 
the coverage according to EOF levels, also indicating how 
the four qualifications covered differ in this respect. 


The hotel assistant qualifications can be found at all 
EQF and ISCED levels, while there are no bricklayer and 
healthcare assistant qualifications linked to EOF level 5. 
There is no ICT service technician linked to EOF level 
3. Healthcare assistant qualifications are predominant- 
ly linked to EOF level 3 while the centre of gravity for 
ICT service technician qualifications seems lie at EOF 
4. The position of the four qualifications can be further 
determined by looking at their orientation towards the 
labour market and their relationship with occupations. 
Table 7-7 shows that 65 out of 80 qualifications aim to 
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train skilled workers, able to work autonomously in a 
particular occupation, taking on the required responsi- 
bilities. This corresponds with the EQF levelling where 
the centre of gravity of the qualification lies at levels 3 
and — especially — 4. 


Some countries have a higher share of qualifications pre- 
paring for semi-skilled work while others have none. For 
example, three of the four New Zealand qualifications 
analysed are preparing for semi-skilled work, as are all 
three from Ireland, two from UK-England and two from 
the Republic of Korea. The link to the labour market and 
to occupations can also be addressed by looking at the 
way qualifications may be legally required to operate in 
an occupation (Table 7-8). 


Table 7-5. EOF levels (actual and estimated) and ISCED 2011 classification 


ESTIMATED EOF LEVEL TOTAL NUMBER ISCED 2011 TOTAL NUMBER 
2 i] 2 6 


3 19) 
4 32 
5 5 
No estimate possible ly 


NB: n = 80 qualifications. 


Source: Auzinger; Broek; Luo mi-Messerer, K., 2017 (forthcoming). 


Table 7-6. EOF levels (actual and estimated) 


3 2 
4 15 
5 S 
No estimate possible 24 


CLASSIFICATION a: EQF LEVEL INDICATION (ACTUAL AND ESTIMATE) 


Type 

Bricklayer/mason 20 2 
Healthcare assistant 17 1 
Hotel assistant/ 23 1 


receptionist 


ICT service technician 


6 7 0 5 
8 5) 0 iS 
5 cs) 3 6 


NB: n = 80 qualifications. 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 
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Table 7-7. Orientation of qualifications: overview per qualification profile and country 


TYPE SKILLED WORKER TOTAL SEMI-SKILLED WORKER | TOTAL 


Bricklayer/mason AL, AT, BB, BG, CL, CR, DK, NZ, KR 
ES), ell, (lt, (EI, IMME, INL, (tat, 
TN, UK-EN, ZA, ZM 


Healthcare assistant /NF, XG, OR, DIX, IES, Fl, FRI, 11 BB, NZ, ZA, IE, KR, UK-EN 6 
LT, MU, NL, PH 
Hotel assistant/ AMAR Ba BGC laG Rab ke 20) lESKRy NA 3 
receptionist ESe EP ERA ER MURNIES NZ 
PH, TN, UK-EN, WS, ZA, ZM 
ICT service technician AMAR BES BGrCRDG@ES 16 JE, MU, NZ, UK-EN 4 
FF, Flr, iat, (Of, INIL, rb, WANE, 
ZM, ZA 


i) 


NB: n = 80 qualifications. 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 


Table 7-8. Function/role of qualifications in the labour market: overview by qualification profile and country 


THE QUALIFICATION TO PRACTISE THE RELATED THE QUALIFICATION 
IS REQUIRED TO dn (@) gl =eote) (0) AND) DO) \\ Ke) IS DESIRED/ 
PRACTISE A CERTAIN THE QUALIFICATION, FURTHER RECOMMENDED 
PROFESSION (LICENSING/ REQUIREMENTS NEEDTO BE BUT NOT A FORMAL 
ACCREDITATION) FULFILLED (CERTIFICATION/ REQUIREMENT 
REGISTRATION) 
Bricklayer/ 1 (DK) 3 (NL, TN, ZA) Gn (BBAINGaC Ree Glee OF 
mason PH, KR, ZM, AT, BG, UK- 
EN, (Fl, (AR, 0, ES, AL) 
Healthcare 7 VAL XG, IDIK, IES, JF, IXR, (EN) 1 (ZA) 9 (UK-EN, FR, IE, NL, BB, 
assistant NZ, CR, MU, PH) 
Hotel 2 (DK, LT) 1 (TN) 201 BiBRNZGeA ANN SMG Re 
assistant/ ZIG EIN Ave Ele 
receptionist KR, AT, BG, UK-EN, FI, 
Fairy, Wile, (NIL, ES, IMIS) 
ICT service 2 (I, WANE) 1 (KR) ‘7 (BiB, INZ, ZN, CIR, ZAM, 
technician MU, PH, AT, BG, DK, UK- 


AN), Al, Ary, Ie, INIL, 1S, NU) 


Fonsi se 


NB: n = 80 qualifications. 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 
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Table 7-9. Purpose of qualifications for further education by qualification profile and country 


NO DIRECT ACCESSTOA ACCESSTOA ACCESS 

ACCESSTOA >. @ AV) Be) 3 NEXT LEVEL OF TO HE 

NT >@ {5 ie) EDUCATION AND EDUCATION AND 

EDUCATION AND TRAINING (BUT NOT TRAINING AND TO 

TRAINING TO HE). HE 
Bricklayer/ 2 (ZM, KR) 11 (AT, BG, UK-EN, FR, 3 (Fl, ZA, PH) PDS, 1a) | 2 ACR, 
mason NL, ES, BB, NZ, TN, CL) 

MU, AL) 

Healthcare 2 (BG, LT) 7 WAI, DIS, NIL, IES, BB, 6 (NZ, KR, PH, UK-EN, - DACh, 
assistant ZA, MU) ll, We) FR) 
Hotel 1 (ZM) 9 (AT, DK, NL, BB, NZ, 9 (BG, UK-EN, FI, FR, DE IES) GR 
assistant/ WS, TN, KR, MU) IE, ZA, PH, NA, MK) (LL) 
receptionist 
ICT service - 9 (AT, DK, NL, BB, NZ, 7 (EG, (Fl, (AR, WE, AML, 3 (UK-EN, 1 (CR) 
technician ZA, UAE, MU, AL) KR, PH) aes) 


fons sr 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 


The data show that most qualifications are desired by oc- 
cupations but that few are (legally) required (7°). 


The orientation of qualifications can also be determined 
by addressing their link to further education, training and 
learning, including links to higher education. It is some- 
times asserted that TVET qualifications are ‘dead-ends’, 
not allowing learners to move to further and higher educa- 
tion and training which, in turns, affects the attractiveness 
of TVET. Table 7-9 questions this assertion, showing that 
only five qualifications provide no direct access to the next 
level of education and training. We also see that a signifi- 
cant proportion of the qualifications also provide access to 
(general) higher education. The importance of advanced or 
higher vocational education and training (outside the uni- 
versity sector) is implicitly underlined by the table, pointing 
to this as an important progress route. 


The contextual data provide an interesting comparative 
picture of the four qualifications. While diverse and cov- 
ering a relatively wide range of levels and trades, the 80 


(2) These are preliminary findings that need to be further checked 
and confirmed, in particular regarding the qualifications defined as 
being legally required. 


qualifications share several common features, irrespective 
of the country of award. 


7.3.2. Qualifications content and profile 

Qualifications have been mapped to the ESCO terminol 
ogy on occupation-specific skills and competences and 
cross-sectoral and transversal skills and competences. Fig- 
ure 7-3 illustrates the extent to which national qualification 
profiles (as a combination of occupational and transversal 
skills) match those used by ESCO. While this does not say 
anything about how the specific content of qualifications 
varies and their relevance to the needs of the labour mar 
kets, It illustrates the variation between countries. 


From this comparison we can distinguish between coun- 
tries with narrower or broader qualification profiles: if we 
look at the occupational-specific skills and competences, 
Spain and Finland demonstrate an almost complete match 
with the ESCO profiles. Albania, Bulgaria and Ireland, in 
contrast, demonstrate a weak match with ESCO. While we 
can observe some significant differences between coun- 
tries in terms of match with ESCO, there are relatively lim- 
ited differences across the four qualification studies. The 
qualification with the best match is the ICT service techni- 
cian, where one third of the countries cover all ESCO skills 
and competences. With the healthcare assistant qualifica- 
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Figure 7-3. Match of national qualifications profiles to ESCO (combining occupation-specific and transversal 
skills and competences) 


——— Hotel assistant/ receptionist =——— Bricklayer/ mason 


UK-EN ” TN 


M AT MU 
MK 100% CL AL 100% CL 
=——— ICT service technician =——— Healthcare assistant 
ucen °F pp 


LT 
ZA 
ZA 
IE 
UAE KR 
FR 
= PH 


80% 
ES oe AT 
DK 100% BG 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 
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tion we find the lowest average match. The data indicate 
that the two service sector oriented qualifications demon- 
strate the lowest match to ESCO and also vary most be- 
tween countries. A better match is demonstrated for both 
the ICT and bricklaying qualifications, possibly reflecting 
the fact that these operate in areas with clearly defined 
technologies and standards, organisation of work and pro- 
essional practices. 


For transversal skills and competences, Spain and Finland 
again demonstrate an almost full match with the ESCO 
terminology (although no country has 100% coverage). 
The highest average percentage of coverage can be ob- 
served for the hotel assistant/receptionist which is slightly 
higher than the figure for the other qualifications profiles. 
The lowest coverage (below 50%) is indicated for five 
qualifications: the bricklayer in Bulgaria (31%) and Chile 
(33%), the hotel assistant/receptionist in Ireland (46%) and 
the healthcare assistant in Korea (88%). The comparison 
across qualification profiles shows that language compe- 
tence Is more important for the hotel assistant/reception- 
ist than for the other qualification profiles, which is under 
standable given the nature of the job. Learning outcomes 
related to numeracy/mathematics are more often included 
in bricklayer/mason and ICT service technician qualifica- 
tions while items related to health and safety seem to be 
less relevant for the ICT service technician qualifications 
compared to the other profiles. 


Transversal competences related to the areas of think- 
ing skills and competences and social skills and compe- 
tences (see previous sections) vary little between the 
different qualifications and also across countries, indicat- 
ing that these are considered relevant across the board. 
Only three of the 81 qualifications covered demonstrate 
a full match with ESCO: the hotel assistant/receptionist 
from the Philippines and the ICT service technician from 
Mauritius and Spain. The lowest coverage (below 60%) 
is indicated for the hotel assistant/receptionist from Bul- 
garia (29%), the bricklayer from Chile (48%) and from 
Bulgaria (51%), the hotel assistant/receptionist from Ire- 
land (55%), and the ICT service technician from Bulgaria 
(56%), the Netherlands (58%) and Barbados (59%). In 
most qualifications analysed (55%), a clear focus on job/ 
occupation-specific learning outcomes can be observed 
but in many (45%), there is an equal balance between 
these types of learning outcomes. An equal balance is 
slightly more prevalent for healthcare assistant qualifi- 
cations and for newer qualifications than for the others. 
However, this trend is not very pronounced. 


VOLUME | — THEMATIC CHAPTERS 


While the distinction between broad and narrow qualifica- 
tions profiles is interesting and indicates that countries de- 
fine their qualifications in different ways, real added value 
is linked to the detailed comparison of which occupational 
and transversal skills and competences have been cov- 
ered. The strength of the ESCO terminology lies in its de- 
tailed approach, operating at a level of granularity making it 
possible to analyse which specific skills and competences 
are being addressed by each qualification. This approach is 
demonstrated by the following two tables (Table 7-10 and 
Table 7-11), showing how the 21 national bricklayer qualifi- 
cations covered by the study match the occupation specif- 
ic and the transversal skills and competences. 


The overviews provided by these tables are unique in the 
sense that they provide a direct and detailed comparison 
of the intentions of national qualification authorities. Using 
the bricklayer qualification as an example, we can immedi- 
ately detect several critical issues. 


Occupation-specific content 
The most important findings regarding occupation-specific 
content can be summarised as follows: 


(a) there is a common core of occupation-specific skills and 
competences; while we have distinguished between 
broad and narrow qualification profiles, among the 21 
countries studied (across the world), a strong common 
core exists. Looking at the total 48 occupation-specitic 
skills and competences listed by ESCO for bricklayers, 
27 are covered and shared by more than 18 countries. 
While not surprising in an area characterised by long and 
strong skills traditions, awareness of this common core 
could aid cooperation between countries, facilitate trans- 
fer and recognition of qualifications, and support learner 
and worker mobility across borders; 
there are different occupational expectations; 10 of the 
occupation-related skills and competences are shared 
by fewer than 15 countries. Half of these are related to 
the management and organisation of working process 
activities. Skills such as ‘ordering of construction stock’ 
and ‘keep personnel administration’ are not prioritised 
by a significant number of countries. This may indicate 
different perspectives on what a bricklayer is expected 
to know, be able to do and understand. Should tasks be 
limited to ‘putting brick on brick’ or should broader skills 
be encouraged and developed; for example manage- 
ment of equipment, team and neighbouring trades? 
(c) while a strong common core can be observed, a few 
countries stand out as operating with significantly differ 


(b 
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ent profiles. For occupation-specific skills Albania, Bul- 
garia, Chile and Denmark operate with clearly narrower 
profiles than most others. 


Cross sector and transversal skills and competences 
The most important findings for transversal skills and com- 
petences can be summarised as follows: 


(a) acommon core of cross-sectoral and transversal skills 
and competences can also be identified: half of the 
transversal skills and competences listed by ESCO are 
covered by 18-21 countries. This points to the existence 
of a strong core of broader skills and competences, 
being combined in most countries with the occupa- 
tion-specific skills; 

there are different degrees of integration of transversal 
skills and competences: the study, in the case of brick- 
layers, shows that countries differ significantly in the 
attention paid to transversal skills and competences, 
with 16 of the 38 skill listed are shared by 15 countries 
and below. It is difficult to point to particular categories 
of skills and competences not addressed; the lack of 
focus might reflect that priority is given to occupation- 
al-specific skills; 

there are significant national differences; three coun- 
tries — Bulgaria, Chile and South Korea - stand out as 
having a significantly different approach to transversal 
skills and competences than the majority of countries 
covered and compared. These countries lack between 
19 and 28 of the total 38 transversal skills and compe- 
tences listed by ESCO. 


Ss 


(c 


While only related to one of the qualifications, the above 
points illustrate the potential of the comparative approach 
in terms of identifying similarities and differences. The 
methodology does not aim at ranking countries. It should 
instead be used by national stakeholders to reflect on the 
priorities set nationally and whether the choices of other 
countries might be of interest. 


7.4. Concluding remarks and issues 


The 80 profiles compared reveal a range of shared charac- 
teristics among the countries’ qualifications for the same 
occupation. This could benefit education institutions and 
companies working at international level, and aid decisions 
on recognition and transfer of individual qualifications. 
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Common elements include: most of the qualifications 
have been reviewed in the past six years; they operate at 
what can be understood as EOF levels 3 and 4; their aim 
is to capture and communicate the education and training 
of skilled workers; and they are normally understood as a 
desirable (even if not legally required) for access to and 
practising an occupation. Most of these qualifications give 
access to further education and training and, in some cas- 
es, also to higher education. 


However, this comparative approach, illustrated by the 
bricklayer case, raises several questions for further work 
and research: 


(a) how accurate are the national qualification profiles; how 
can we work to validate further the data underpinning 
the comparison? 

(b) can the ESCO approach, given publication of a quality-as- 
sured official version in 2017, be used as a basis for fu- 
ture comparative work? 

(c) who, at national and international levels, could be in- 

volved to validate further and strengthen the data? 

how can these comparisons, when strengthened and 

stabilised, support national and international policy de- 

velopments related to qualifications? 

how will digitalisation, including digital learner records 

and credentials, affect the way we access data and use 

it for faster and broader comparison. 


(d 


(e 


The study will be followed up by Cedefop, with particular 
focus on the processes adopted by the countries to review 
and update their qualifications. The ETF Cedefop and UN- 
ESCO are actively discussing the continuation of their col- 
laboration to ensure that this global perspective on TVET is 
further developed and strengthened. 
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Table 7-10. Job/occupation-specific KSC: bricklayer 


check straightness of brick 


fill up mortar joints 

finish mortar joints 

follow health and safety procedures in construction 
lay bricks 

secure working area 

use measurement instruments 

use safety equipment in construction 

work safely at heights 

keep workplace tidy 

maintain equipment 


work in a construction team 
interpret 2D and 3D plans 

mix construction pastes and grouts 
sort waste 

split bricks 


calculate needs for construction supplies 
mix concrete 

inspect construction supplies 

apply finish to concrete 


install construction profiles @ 


transport construction supplies 
work ergonomically 


building codes 
apply damp-proofing and waterproofing membranes e 
build scaffolding 

operate masonry power saw 


remove concrete forms 


apply restoration techniques 
inspect supplied concrete 
place concrete forms e 


pour concrete C 
screed concrete 


install falsework e 
install insulation material c 
keep records of work progress 

reinforce concrete 

rig loads 


snap chalk line 


monitor stock level 
operate surveying instruments 


process incoming construction supplies 


set up temporary construction site infrastructure 


use squaring pole @ 


estimate restoration costs e @ 
keep personal administration 
document survey operations 


order construction supplies @ ® 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 
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Table 7-11. Transversal KSC: bricklayer 


follow hygienic work practices 


carry out work-related measurements 


work with shape and space 


apply quality standards 
support company plan 
manage time 
memorise information 


follow safety precautions in work practices 


communicate mathematical information 
evaluate information 
make decisions 


process qualitative information 


handle quantitative data 


accept constructive criticism 

lead others 

support colleagues 

develop strategy to solve problems 
carry out work-related calculations 


interact with others 


use mathematical tools and equipment 


report facts 


work in teams 

use learning strategies 
motivate others 
recognise opportunities 


think creatively 


follow environmentally sustainable work practices 


support cultural diversity 

instruct others 

demonstrate intercultural competence 
give advice to others 

negotiate compromise 

persuade others 

use body language 


address an audience 


use questioning techniques 


support gender equality 


mother tongue 


foreign language 


NB: n = 81 qualifications. 
@: KSC terms that are either explicitly or implicitly covered. Remaining KSC terms are not covered at all. 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 
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LEARNING OUTCOMES WORLD REFERENCE LEVELS: GLOBAL 
SOLUTIONS FOR GLOBAL CHALLENGES 


Borhene Chakroun and Katerina Ananiadou, UNESCO 


Summary 


The Third international congress on technical and vocational 
education and training (TVET), held in 2012, recommended 
the development of international guidelines on quality assur 
ance for the recognition of qualifications. This would be based 
on learning outcomes and the identification of a set of world 
reference levels (VWWRLs). The main purpose of this work is to 
increase transparency of qualifications and aid internation- 
al comparison and recognition of TVET qualifications. UN- 
ESCO, in collaboration with experts, international partners 
and regional organisations overseeing regional qualifications 
frameworks, has been leading this work using a multistage 
approach. This chapter provides an update on progress and 
challenges related to the establishment of WRLs. It presents 
an overview of the main findings of a set of studies: a com- 
parative study of qualifications cross-borders; an analysis of 
level descriptors and a preliminary structure of WRLs; and a 
study on the way qualifications frameworks relate to each 
other with a particular focus on referencing processes. 


Keywords: qualification; world reference levels (VWRLs); 
referencing; level descriptors 


8.1. Introduction 


In 2012, UNESCO convened the Third international con- 
gress on TVET in Shanghai to debate current trends and 
future drivers of education and training development. This 
global dialogue culminated in the Shanghai consensus, 
which recommended developing international guidelines 
on quality assurance for the recognition of qualifications 
based on learning outcomes ('). This included the proposal 
that a set of world reference levels (VW/RLs) be considered 
to support international recognition of TVET qualifications. 
Since 2014, UNESCO has been working towards develop- 


(") Shanghai consensus: recommendations of the Third international 
congress on technical and vocational education and training, Trans- 
forming TVET: building skills for work and life. Shanghai, People’s 
Republic of China, 14 to 16 May 2012. http://www.unesco.org/file- 
admin/MULTIMEDIA/HO/ED/pdf/concensus-en.pdf 


ing such guidelines, in partnership with regional and glob- 
al organisations. As discussed by Chakroun and Daelman 
(2015), UNESCO has adopted a fourstaged incremental ap- 
proach to this work: a technical review of level descriptors 
at national and regional levels; conceptual development of 
the WRLs; broad consultation; and a political process that 
will explore the technical and legal aspects relating to the 
desirability of defining and adopting WRLs. 


Approaching the work in a bottom-up way, a range of re- 
views and consultations have been conducted since 2015, 
bringing together many international experts in qualifica- 
tions and qualifications frameworks. This chapter reports 
on progress made since 2015 in terms of advancing the 
conceptual development of WRLs. 


8.2. The purpose of the world reference levels 


The most important factor behind developing WRLs is the 
internationalisation of education and training systems and la- 
bour markets and the increased mobility of people and jobs. 
The existence of regional frameworks and the international 
visibility of national qualifications are additional stimuli. The 
added value of the world reference levels lies in their poten- 
tial to provide an independent reference point against which 
a level of learning outcomes can be compared internation- 
ally. The levels should broadly aim at supporting learner and 
worker mobility, and participation in the labour market and 
lifelong learning. They should support equity of recognition 
by including quality assurance principles, while addressing 
the challenges of interregional mobility through capaci- 
ty-building, development of a common language, shared ori- 
entation materials and the establishment of networks and 
communities of practice. The world reference levels should 
complement national qualifications frameworks (NQFs) 
and regional qualifications frameworks (ROFs), and support 
monitoring of regional and international development. 


WRLs should not be seen as a set of standards, in the sense 
of requirements, or a type of world qualifications frame- 
work, but rather as a translation tool, that can support ‘glob- 
al mobilities’. They should allow the primary descriptions of 
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any system to be re-expressed using a global lexicon. The 
advantage of this is the ability to offer a neutral, independent 
and international reference point against which the levels of 
outcomes expressed in different forms can be compared 
and to which qualifications can be pegged. They should 
provide a point of comparison for international/interregional 
dialogue about learning outcomes and qualifications, their 
value in entry to formal education and training, and entry to, 
or progression in, employment. They should be capable of 
being used not only with national, sectoral and regional qual- 
ifications frameworks, but also with other outcomes-based 
constructs, such as job search and career guidance frame- 
works, job evaluation and recruitment schemes, and other 
sectoral national, and international structures. 


8.3. Current initiatives and main findings 


Since 2015, UNESCO has made progress in the concep- 
tual development of the proposed WRLs by undertaking 
several research studies in collaboration with international 
experts in the field. The following studies have been car 
ried out: a comparative study of qualifications cross-bor 
ders; an analysis of level descriptors; and a study on the 
way qualifications frameworks relate to each, other with a 
focus on referencing processes. The objectives, method- 
ology, key findings and conclusions of these three studies 
are presented in detail in the next sections. 


8.3.1. Comparative study of qualifications (?) 


Introduction 

The purpose of this study was to provide empirical support 
in understanding how TVET qualifications are defined, re- 
viewed and renewed and how these processes influence 
their profile and content. Using comparative methodology, 
the study aimed to: 


(a) analyse and compare the description of qualifications in 
terms of learning outcomes in a systematic way; 

(b) analyse and compare the categories used for describ- 
ing qualifications. 

It was anticipated that the systematic demonstration of 

similarities and differences among qualifications would al- 

low identification of the most frequently used categories 

and items. This, in turn, could be used as a basis for an 


(4), This work was conducted by Monika Auzinger, Simon Broek, 
Karin Luomi-Messerer, in collaboration with Cedefop, the ETF 
and UNESCO. 
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international common template, a minimal common struc- 
ture for describing qualifications. 


The study focused on four qualification profiles available 
globally and capable of being compared in terms of their 
scope, content and structure: bricklayer/brick mason; 
healthcare assistant; hotel assistant/receptionist; and ICT 
service technician. 


The study covered 38 qualifications from 26 countries dis- 
tributed geographically as follows (°): five in Africa, two in 
Asia, 12 in Europe, one in the Gulf region, three in Latin 
America and the Caribbean and two in the Pacific. Coun- 
tries represented different income levels and_ national 
qualifications systems, covering the span of what is usu- 
ally called the three generations of national qualifications 
frameworks (Coles et al., 2014). All countries involved have 
in common an effort to introduce learning-outcomes-based 
national qualifications frameworks. 


Methodology 

The detailed methodology and results of the comparative 
study are presented by Bjarnavold and Chakroun in Chap- 
ter 7 of this publication. This chapter focuses on the analy- 
sis conducted by the experts to identify the most frequent- 
ly used categories for describing qualifications. 


A template was prepared for analysing the structure of 
qualifications, the qualification description categories. This 
template was then used for indicating whether information 
on specific items is included in the descriptions. The assess- 
ment was based on official descriptions of qualifications (for 
example, as presented in national databases or NOFs). The 
template was developed drawing on the available literature. 
It included categories or data fields used for the Europass 
certificate supplement (*) and those suggested by the Euro- 
pean qualifications framework advisory group for present- 
ing information on describing qualifications with a European 
qualifications framework (EQF) level in the context of the 
Learning opportunities and qualifications in Europe portal (°). 


Findings 

The qualification items can be broadly clustered into five 
main areas: basic information about the qualification; as- 
sociation with international/national categorisations/classi- 


(°) Only part of the global study is reported here. 

(*) https://europass.cedefop.europa.eu/en/faq/certificate-supple- 
ment-questions 

(°) http://ec.europa.eu/ploteus/en/home 
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fications and frameworks of qualifications; official basis for 
the qualification; acquiring the qualification; and profile and 
scope of the qualification. 


The findings are summarised in Table 8-1. The catego- 
ries highlighted in a darker shade could be used as a 
starting point for developing common templates for de- 
scribing qualifications. 


Conclusions 

Using the most frequently used categories for describing 
qualifications, the following categories could be a starting 
point for developing a common template for describing 
qualifications internationally: 


(a) title of the qualification in English; 

(b) country/region; 

(c) name of the body awarding the certificate; 

(d) type/form of assessment; 

(e) learning outcomes descriptions; 

(f) scope of the qualification (full, partial, special pur 
pose qualification); 

(g) thematic area/economic sector; 

(h) NOF level (°). 


Further information on qualifications beyond the items 
presented above would improve their wider transparency 
across the world. Additional data fields that can aid the use 
of the information for recognition of qualifications and help 
users to understand the quality level and trustworthiness 
of qualifications would achieve this purpose. For instance, 
additional optional items could be considered including the 
geographic dimension (national/international), the orienta- 
tion of the qualification (academic/professional/mix) and 
accreditation and other quality assurance processes. 


8.3.2. Referencing processes (’) 


Introduction 

Referencing processes and methodologies are increas- 
ingly being used when qualifications frameworks are 
compared (Keevy and Chakroun, 2015). The WRLs expert 


(°) The last two items are not among those most commonly used in 
the 39 countries analysed in this study (as presented in Table 8-1). 
We suggest that they should be considered for a minimal common 
structure of a template for describing qualifications because they are 
strongly emphasised in the European context. They belong to the 
‘required data fields’ suggested in the EOF context for the common 
format for the electronic publication of information on qualifications. 

(’) This work was carried out by Diane Booker (Booker, 2016). 
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group identified this as a key area to be analysed to de- 
termine the implications for the proposed world refer 
ence levels. The preliminary outcomes of this work are 
presented below. 


Methodology 

The work included mapping referencing processes 
throughout the world, including techniques, methodol- 
ogies used, elements of organisation, quality assurance, 
actors involved, highlighting comparable elements as well 
as differences, and examining the outcomes of these pro- 
cesses and their impact. 


The study adopted the use of the term ‘referencing’ based 
on common use in Europe and other parts of the world to 
describe the process of comparison of national qualifica- 
tions frameworks with a regional qualifications framework 
or another national qualifications framework. Adopting the 
term for this work was done in the knowledge that refer 
encing has a particular meaning in the European context in 
relation to the EOF and that it may not necessarily trans- 
late well in the global arena. 


The EOF considers referencing as an agreed ‘process that 
results in the establishment of a relationship between the 
levels of national qualifications, usually defined in terms of 
a national qualifications framework, and the levels of the 
EQOF' (or the levels of another local (°), national or regional 
qualifications framework) (EC, 2013). It is also widely ac- 
cepted in the EU that ‘the referencing process involves 
each country ... describing its qualifications system, quality 
assurance arrangements and reporting the extent to which 
the country meets the (agreed referencing criteria) (°) that 
structure and quality assure the referencing process’. An 
important part of this process is the involvement of all 
stakeholders in the qualifications system to ensure objec- 
tive and external scrutiny of national systems, the process 
and the referencing outcomes (Keevy et al., 2016, p. 19). 


The term ‘referencing’ is commonly used to describe the 
process of comparing national qualifications frameworks 
with a regional qualifications framework, such as has 


(°) For instance, Hong Kong is a Special Administrative Region of the 
Peoples’ Republic of China and as such the Hong Kong qualifica- 
tions framework (HKOF) is described as a local framework and its 
referencing activities are examples of referencing a local frame- 
work with other national or regional frameworks. 

(°) The EQF describes 10 criteria; referencing between other coun- 
tries may have agreed on other criteria although most are derived 
from the EOF criteria. 
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Table 8-1. Categories for describing qualifications: most frequently used categories 


ITEM/CATEGORY DETAILED DESCRIPTION FREQUENCY 
OF ITEM USE 


1. Basic information Title of the qualification St) 
about the 
qualification 

Country/region in which the qualification is awarded 34 
2. Association Thematic area — international classification: ISCED/ fields of education 8 
with international/ and training 
national 
categorisations/ Economic sector — international classification: NACE code 3 
classifications and ; ; ; are . 

Thematic area/economic sector — national classification: national 28 


frameworks of 


qualifications classifications for clustering programmes related to their thematic or 


sectoral focus 


Level — international classification: ISCED level 7 
Level — national classification: NOF level 29 
Level — national classification: countries might use other (national) 2S 


systems for classifying their education programmes or qualifications 


3. Official basis for Legal basis of the qualification 21 
the qualification 
Name of the body awarding the certificate 35 
Name of the national/regional authority providing accreditation/ 29 


recognition of the certificate 
International recognition agreements 2 


Grading scale/Pass requirements: the grading scale informs about the 33 
standardised measurement applied to identify the levels of achievement 

in an education programme. The pass requirements indicate the level of 
achievement necessary for passing a course 


4. Acquiring the Official recognised ways of acquiring the qualification, for example, the 28 
qualification type of programme leading to the qualification (e.g. school-based or 
work-based pathway/programme) or to the type of exam required for 
obtaining the qualification 


Type of providers/institutions offering the pathway towards obtaining the 24 
qualification 


Entry requirements: this refers to the requirements that need to 30 
be fulfilled for entering the pathway or gaining access to a training 

programme leading to the qualification (such as requirements related to 

age, previous qualifications, work experience) 


Volume/duration of learning required: for example, number of learning 27, 
hours or years of training necessary for obtaining a qualification; 
sometimes expressed in credit points 


Extent of company-based training: this specifies the amount of 23 
workplace training required 


Accreditation and other quality assurance arrangements of the training 22 
programme leading to the qualification 
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ITEM/CATEGORY DETAILED DESCRIPTION 
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FREQUENCY 


OF ITEM USE 


Arrangements for the validation of prior learning: processes for 18 
evaluating learning outcomes which could result, for example, in 
exemption from parts of programmes or exams 


Type/form of assessment 


5. Profile and scope 


of the qualification qualifications or to parts of it 


33 


Credit points: credit points indicate the value assigned to a programme/ 24 


Description in terms of learning outcomes 3g 


Range of occupations accessible to the holder of the qualification: 28 
this refers to the occupations certificate holders may pursue or any 


entitlements in the labour market 


Main target groups of the qualification: characteristics of learners 30 
expected to obtain the qualification, e.g. in terms of age group, work 


experience, or employment status 


Geographic dimension: indication of the labour market context for which 16 


the qualification prepares 


Orientation of the qualification: for example, general education, 25 
vocational/professional education. academic education or a mix 


Scope of the qualification: size and purpose of a qualification, e.g. 28 
whether full, partial, or special purpose 


Source: Auzinger; Broek; Luomi-Messerer, K., 2017 (forthcoming). 


been the practice in Europe since the introduction of the 
EOF recommendation. Increasingly, as countries are con- 
sidering comparison of individual national qualifications 
frameworks with each other (or even just sections of the 
frameworks), different terminology such as ‘benchmark- 
ing’, ‘self-certification’, ‘alignment’ and ‘comparison’ are 
being used. New Zealand suggests that comparison is 
the overarching term and New Zealand's use of compar 
ative analysis approaches enables inclusion of a greater 
level of detail in the mapping (pathways, credits, inputs, 
outputs). Analysis of level descriptors expressed in terms 
of learning outcomes used in each framework is a critical 
component of this comparison process. 


Table 8-2 compares the terminology used in different con- 
texts. Irrespective of the particular term used, it appears 
that the outcome of a referencing process is the devel- 
opment of mutual trust between countries, increased 
understanding of qualifications systems and their legal 
basis, and definition of the relationship or correspond- 


ence of the levels of the national/regional qualifications 
systems to improve transparency and aid international 
recognition of qualifications. 


Findings 
Several relevant referencing processes of qualifications 
levels have been identified: 


(a) the referencing of national qualifications levels to region- 
al qualifications levels (and particularly to the EOF both 
by European and non-European States); 

the referencing of national qualifications levels to anoth- 
er country’s set of national qualifications levels, such as 
Malaysia with New Zealand; 

examples of countries referring their NOFs to other 
countries with an objective to benchmark levels and lev- 
el descriptors; 

other forms of referencing, such as sectoral interna- 
tional qualifications, including welding, tourism, hospi- 
tality, banking. 


(b 


(c 


(d 


118 


Table 8-2. Terminology related to referencing processes 


TERM DEFINITION ACTION/OUTCOME CONTINUUM 


Alignment 


Benchmarking 
(study) 


Comparability (study) 


Comparative 
(analysis) 


Compare 


Comparison 


Compatible 


Compatibility 


Mapping 


Reference point 


Referencing (process) 


Relationship 
(analysis) 


Relative (to) 


Source: Booker (2016). 


Agreement, alliance or cooperation among 
persons, groups. 


Test or measure something against a stand- 
ard (or develop the standard) UAE: means 
the continuous process of measuring and 
comparing products, services and practices 
with comparable systems or organisations 
both inside and outside the UAE for the 
purpose of continuous improvement 


Capable of being compared; features in 
common 


Using comparison as a method of study 


Liken, similar to 


Considering the common characteristics 
between two or more ‘things’ 


Used to establish a system-to-system 
level agree ability or harmony of national 
qualifications frameworks level descriptors 
and qualifications definitions (New Zealand 
glossary) 


Consistent with something else, 
agreement, correspond, in accord 


Delineation, representation, reflection 


Basis or standard for evaluation, 
assessment or comparison (Shorter Oxford 
third edition) 


Process that results in the establishment 
of a relationship between the levels of 
local, national or regional qualifications 
framework) (EC, 2013 p. 6). 


Connection, association, involvement 
between parties; state or mode of being 
related/connected (Shorter Oxford third 
edition) 


Reference to, relating to (Shorter Oxford 
third edition) 
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Implies a political agreement (e.g. Shorter 
Oxford third edition — fall into line with) 


Implies an unequal relationship: one side 
has set the standard against which the 
other is measured 


HK/EOF: analyses the technical and 
conceptual characteristics of the respective 
frameworks in the context for which they 
are designed and, by comparing the two 
frameworks, seeks to identify key aspects 
of similarities and differences and thereby 
determine the comparability between them. 


Designating the degree of comparison 
(Shorter Oxford third edition) 


Consider or estimate the similarity or 
dissimilarity of one thing to another 
(Shorter Oxford third edition) 


Action or an act of likening or representing 
as similar (Shorter Oxford third edition) 


Used in NZ/Ireland report 2010 


Lower level type of comparison 


Used in EOF terminology 


EU context is a political requirement 


Thesaurus: comparable, related 
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The impacts and benefits of referencing are difficult to 
measure but there appears to be evidence that referenc- 
ing outcomes can build confidence in a country's qual- 
ifications and associated system. The European Com- 
mission has found that one key impact is an increase in 
motivation to develop, implement and strengthen NOFs 
in most countries (European Commission, 2013). Ben- 
eficiaries of the referencing process include learners, 
employers and employees, educational institutions, in- 
dustrial relations organisations, accreditation bodies and 
other stakeholders. 


There is no single model for the referencing process but 
EQF criteria and processes have provided a significant 
normative influence on referencing activities elsewhere. 
There is a strong trend towards increased referencing 
between countries and between regions, with the most 
‘active’ country being New Zealand. There is, however, 
no clear model for referencing outside the formal edu- 
cation and training system, such as for sectoral interna- 
tional qualifications. 


Conclusions 

The work conducted so far compares the different ter 
minology used and includes a definition and preliminary 
range of outcome statements. While EQF criteria and 
processes have provided a significant influence on ref- 
erencing activities in other parts of the world, there is a 
need to draw on other experiences and to work on guide- 
lines for other forms of referencing, including sectoral in- 
ternational qualifications. 


Based on the above analysis, it is proposed that referenc- 
ing processes in the context of WRLs should be: 


(a) transparent; 

(b) based on robust quality assurance processes; 

(c) evidence-based; 

(d) representative of stakeholder views; 

(e) objective to ensure trust; 

(f) based on useful, practical outcomes; 

(g) ensure accessibility of outcomes; 

(h) dynamic and subject to continuous review and revision. 


These guidelines, and the outcomes of this work more 
generally, feed directly into the development of the model 
WRL presented in Section 8.3.3, particularly with regard to 
the development of factor-descriptor statements for each 
stage and the table of concordance. 
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8.3.3. International review of level descriptors ('°) 


Introduction 

In 2015, Keevy and Chakroun (2015) conducted a first 
analysis of the use of level descriptors based on learning 
outcomes. A broad range of level descriptors was ex 
amined, drawing on national and regional qualifications 
frameworks, as well as longitudinal studies, international 
competence assessments and diagnostic review. The au- 
thors found that some form of knowledge, skills and com- 
petences were the most widely used domains, and should 
be considered as the most appropriate for the world ref- 
erence levels. It was found that all three domains can be 
described using learning outcomes, or sets of learning out- 
comes. Linking the descriptor phrases within levels and 
across domains to form whole sentences (as is done in the 
Association of Southeast Asian Nations (ASEAN) regional 
qualifications framework) the need to add exemplars from 
various countries and regions, and indicative levels were 
also noted as important considerations for the world ref- 
erence levels. 


Methodology 

This section presents the results of a study that builds 
on the work of Keevy and Chakroun (2015) on the use of 
level descriptors. It provides a comparative analysis of the 
structures, terms and concepts used for defining the level 
descriptors, and identifies and compares the factors used 
to characterise levels and create the level-to-level steps 
in and across domains. The aim is to identify significant 
commonalities and understandings around the world, and 
reveal common conceptual bases for setting levels and 
progression. The analysis should also flag any identifiable 
changes in the uses of level descriptors. 


The study was conducted as follows: 


(a) methodology was developed for comparative analysis of 
15 national and regional qualifications frameworks: (") 
the national qualifications frameworks of Australia, 


(0) This work was conducted by John Hart (Hart, 2016). 

(") Account was also taken of Cole and Oates (2005), Markowitsch 
and Luomi-Messerer (2008) and Winterton et al. (2006). They also 
take into consideration a range of relevant frameworks which are 
not qualifications frameworks, such as the UK National Health 
Service and Social Work frameworks of competences; the UK's 
University for Industry level descriptors; the Hay MSL job evalua- 
tion framework, aspects of the World Bank step programme and 
the O*Net report; mapping undertaken at different times by the 
construction and engineering sectors in the UK; the OF Emirates 
employability indicators. 
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Germany, Indonesia, Ireland, Mexico, New Zealand, 
UK-Scotland, South Africa and the United Arab Emirates 
and the regional frameworks of the ASEAN, the Carib- 
bean Community (CARICOM), the EU, the Pacific, the 
Southern African Development Community and the 
Small States of the Commonwealth; 

(b) findings and recommendations of the initial round of 
analysis were discussed and reviewed by experts. 


Based on the results of the first comparative analysis and the 
experts’ feedback, the report and recommendations were re- 
visited and revised; a draft conceptual framework for the con- 
tent analysis of a larger set of frameworks was then prepared. 
The outcomes of this analysis are set out in Section 8.3.3.3. 


Findings 

The number of levels used in NOFs varies but most have 
eight or 10; some frameworks include additional access lev- 
els. The frameworks in the sample employ between two 
and 10 groupings or headings to structure their level de- 
scriptors. Most, but not all, call these ‘domains’; one frame- 
work has no overt groupings. The format and content of 
descriptors Is also varied along the following dimensions: 
use of words, phrases or sentences; simple, complex or 
detailed; and use of shared headers and bullet points. 


From this analysis, a first conceptualisation for WRLs 
is proposed which emphasises progression and re- 
quires a considered referencing process. It proposes 
a three-dimensional structure based on analysis of 
qualifications frameworks and named in ways which 
distinguish them from such frameworks. The following 
dimensions are proposed: 


(a) a small number of broad stages; 

(b) a range of key factors and markers that are common to 
most qualifications frameworks and used in many of the 
other structures referred to above. These are set out in 
sector neutral language, using a small number of terms 
to discriminate between stages; 

definitions and explanations of these factors and markers; 
a series of concordances of these factors and markers 
that set the stages. They will work by linking the terms of 
the WRLs to the terms (including sectorspecific terms) 
used in qualifications frameworks. 


(c 
(d 


Each stage includes a basic level and an advanced level: ba- 
sic stage A and advanced stage A, and so on. This is intend- 
ed to create a level playing field for comparisons between 
frameworks, based on different assumptions about the 
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number and nature of levels. Quality assurance processes 
could be added, but are not included in these proposals. 


Stage A 


The basic characteristics of this stage are the com- 
petences and capabilities for meeting the essen- 
tial demands of active citizenship, basic education 
and participation in a work role: functional literacy, 
numeracy, use of information and communication 
technology, backed by general knowledge and the 
ability to carry out structured tasks and procedures 
and work with others in a familiar context. In many 
countries this is identified with the completion 
of primary education and in some countries with 
‘work readiness’. 


The advanced characteristics at this stage are the 
competences and capabilities associated with a par 
ticular area of study or a specific work role or occupa- 
tional activity, such as ‘assistant worker’ or ‘helper’ or 
‘supervised operative’. It might be seen as the level 
of the first meaningful general or vocational qualifi- 
cations. In some countries this is identified with the 
completion of compulsory education and introductory 
vocational training. 


This stage appears to accord with levels 1 and 2 in 
most frameworks. 


Stage B 


The basic characteristics of this stage are the compe- 
tences and capabilities required for undertaking study 
leading to progression from secondary education or for 
undertaking work as a partially skilled, but qualified, 
worker or an ‘independent operative’. 


The advanced characteristics at this stage are the 
competences and capabilities associated with fitness 
to take on learning, leading to tertiary or higher educa- 
tion or to function as a skilled worker in a trade or craft, 
or to take on supervisory responsibilities. 


This stage appears to accord with levels 3 and 4 in 
most eight-level frameworks, but relates to three lev- 
els in several frameworks. 
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Stage C 


The basic characteristics of this stage are the compe- 
tences and capabilities associated in an education con- 
text with the requirements for learning in short-cycle 
qualifications in the first cycle of tertiary or higher ed- 
ucation and associate degrees, and in a work context 
with technician, specialist or ‘para-professional’ work 
roles, often with some managerial responsibilities. 


Advanced characteristics at this stage are the compe- 
tences and capabilities associated with the descriptors 
for full first cycle degrees and entry to the second cy- 
cle of higher education and in a work role with inde- 
pendent/senior technician, specialist, professional or 
middle manager. 


This stage appears to accord with levels 5 and 6 in 
most eight-level frameworks, but relates to three lev- 
els in several frameworks. 


Stage D 


The basic characteristics of this stage are the com- 
petences and capabilities associated in an education 
context with the requirements for learning in the sec- 
ond cycle of tertiary or higher education and in a work 
context with the requirements for specialists, profes- 
sionals and executive managers. 


The advanced characteristics of this stage are the 
most advanced intellectual, academic, specialist and 
managerial competences and capabilities. Work roles 
include technologists, analysts, and executive profes- 
sionals with extensive and/or strategic responsibilities. 


This stage appears to accord with top levels in 
most frameworks. 


It is proposed that these stages are described with state- 
ments based on the following 11 factors: 


(a) carrying out activities; 

(b) using skills and applications; 

(c) dealing with problems and carrying out research; 
(d) communicating; 

(e) accessing and using data; 

(f) taking responsibility for activities; 
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(g) working with others and giving leadership; 
(h) monitoring and improving quality; 

(i) using and extending knowledge; 

(j) responding to context; 

(k) applying values. 


It is further proposed that each factor should be set out in 
eight statements (two for each stage). These statements 
could take the form: ‘The individual wvill../. This would 
make them more elaborate than the level descriptors used 
in some frameworks such as the EOF, but less elaborate 
than some frameworks which may use two or three cir 
cumstantial sentences. For example, a statement for car 
rying out activities at basic stage A could take the form: 
the individual will carry out tasks and procedures that are 
simple, familiar, routine and/or highly structured. 


It is intended that the WRLs should be accompanied by a 
kind of glossary or concordance table which will set out the 
factors and markers in the WRLs and link them to words or 
phrases used in different frameworks to indicate the same 
idea, characteristic or quality; as example, the VWWRL factor 
term ‘activity’ should be understood to cover terms such as 
task, action, process, project, work, or function. An illustra- 
tive concordance table is currently being developed. 


Conclusions 

The above concepts and definitions are currently being dis- 
cussed and reviewed by the international expert group. The 
group has raised questions and provided feedback on several 
aspects, such as the division of each stage into basic and 
advanced statements. The group also highlighted the need 
to address some of the gaps and inconsistencies identified in 
the factors themselves. The next step for this work will be to 
test these concepts and their definitions to find out whether it 
is technically feasible to develop them further to create more 
coherent and consistent statements, and to identify any gaps 
and inconsistencies. It was agreed that the WRLs will be test- 
ed with a range of outcomes-based structures: these may in- 
clude a level from a sectoral competence framework, a level 
from a sectoral career framework or management structure, 
a national qualifications framework level, a qualification spec- 
ification, and a qualification type specification. 


8.4. The way forward 


The work presented in this chapter is part of the con- 
ceptual development process around WRLs that has 
been led by UNESCO in collaboration with an interna- 
tional group of stakeholders and experts since 2014. The 
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wide diversity in the organisations and bodies involved 
in developing qualifications and using level descriptors 
should not be underestimated. UNESCO is also deepen- 
ing its mapping of the approaches used for ensuring the 
quality of qualifications at national, regional and interna- 
tional levels. Quality assurance is the crucial dimension 
in value and recognition by the labour market. The chap- 
ter on quality assurance of qualifications in this Global 
inventory illustrates the attention given to quality assur 
ance of qualifications. 


In this chapter, we present a first attempt at conceptual- 
ising the world reference levels, drawing on the different 
national and regional frameworks that already exist, as 
well as the wider literature developed in this area over the 
past few years. The conceptualisation also draws on the 
extensive discussions that have taken place in the context 
of the expert group convened by UNESCO on this topic. 
This work is not yet completed and is currently being re- 
viewed by the group of experts; the intention is to trial 
the concepts of stages, factors and concordances on a 
small scale to assess Its usefulness and suitability, and to 
identify inconsistencies and gaps that can be addressed 
in subsequent revisions. The bottom-up approach that 
has been adopted in its development is, therefore, at the 
heart of this process. 


The following initiatives also Support the conceptual devel- 
opment of world reference levels: 


(a) mutual and peer learning and capacity development is en- 
couraged and supported. Substantial learning is already 
taking place through the peer learning exercises conduct- 
ed, for example, between the Southern African Develop- 
ment Community (SADC) technical committee on certifi- 
cation and accreditation and the EOF advisory group; 

UNESCO is currently working towards the development 
of a global convention ('2) on the recognition of higher 
education qualifications. The general aim of this inter 
national normative instrument is to promote interna- 
tional cooperation in higher education, strengthen and 
promote international mobility and lifelong learning, and 
promote the coherence between recognition, quality as- 
surance and qualifications frameworks, while recognis- 
ing the growing diversity in the sector. This activity, and 
the future outcomes of this work on the convention, are 
relevant to the development of WRL, particularly in the 


(b 


(2) http://unesdoc.unesco.org/images/0023/002352/235261e.pdf 
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context of the increased permeability and diversity of 
technical, vocational and academic programmes at the 
post-secondary and tertiary levels. The conceptual devel- 
opment of the global convention on higher education is 
being followed closely by experts working on the WRLs. 


The development of the world reference levels will further 
benefit from focused research in the following areas: 


(a) the use of learning outcomes in level descriptors, curric- 
ula and assessment tools; 

(b) the application of digital technologies in the area of rec- 
ognition of skills and qualifications. 


Work in these areas is already under way and being dis- 
cussed by the expert group convened and led by UNESCO; 
the outcomes of this additional work will be disseminat- 
ed to the broader expert community as soon as they are 
available. This chapter provided a snapshot of the progress 
achieved so far in the conceptual development of WRLs. 
This technical work, despite its challenges and limitations, 
is an essential first step towards the task of developing a 
robust set of world reference levels that could ultimately 
be adopted and used by a highly diverse set of organisa- 
tions, institutions and individuals in an increasingly glo- 
balised and mobile world. 
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CONCLUSIONS 


The 2017 Global Inventory provides some important insights 
into the current development and implementation of quali- 
fications frameworks worldwide. The following points sum- 
marise some of the main trends which can be observed. 


(a) The number of national qualifications frameworks has 


(b 


(c 


stabilised at around 150. These frameworks can be 
found in all regions of the world. 

Most qualifications frameworks were set up during the 
past decade, partly triggered by the development of re- 
gional frameworks such as the European qualifications 
framework (EQF) and the reference framework of the 
ASEAN. This indicates extensive policy borrowing and/or 
policy learning over a relatively short period. A key ques- 
tion is whether these new frameworks are mere ‘policy 
hypes’, destined to fade away, or whether they are being 
turned into integrated parts of national and regional poli- 
cies and systems. 

While the long-term sustainability and impact of quali- 
fications frameworks has yet to be identified on a glob- 
al scale, developments after 2015 show that several 
national qualifications frameworks (NOFs) have found 
their place in the education, training and employment 
policy landscape and are turning into operational enti- 
ties. This is clearly the case in Europe, but can also be 
observed for South-East Asia and other regions. This 
speaks against characterising qualifications frameworks 
as mere ‘policy hypes’. 


(d) The first generation of qualifications frameworks (Aus- 


tralia, New Zealand, South Africa and the UK) are fully 
integrated into national systems but are undergoing 
continuous change and adaptation. The most significant 
change took place in UK-England where the qualification 
and credit framework (OCF) was abolished and, replaced 
by a new, non-regulatory framework. 

Between 2015 and 2017 we can observe a tendency 
towards strengthened regional cooperation between 
NOFs. This is the case in Europe (as demonstrated 
through the adoption of a revised EQF recommenda- 
tion), but can also be observed in Asia where the ASE- 
AN reference framework is now operational. Efforts to 
revive cooperation between frameworks in the South- 
ern African region (SADC) also illustrate this tendency. 
This increase in regional cooperation is important as it 
aids dialogue on cross-border transfer and recognition of 


(f) 


(g 
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qualifications and raises the question of how NOFs can 
support learner and worker mobility. 
Developments in the USA differ from those seen in 
other countries. While the new credentials framework, 
promoted by the Lumnia foundation, pursues many 
of the same (transparency) objectives as others, it is 
organised as a voluntary/private venture and operates 
independently of federal or State authorities and legis- 
lation. The credentials framework confirms the general 
and increasing need for transparency of qualifications 
and credentials. It underlines that qualifications frame- 
works need to be relevant to the needs of individual 
citizens, who face an increasingly complex world of 
qualifications and credentials. 
) The new generation of frameworks (particularly in Eu- 
rope) differs from first generation frameworks by em- 
phasising communication and transparency rather than 
regulation and harmonisation. These frameworks are 
‘loose’ in the sense that they have been designed to 
embrace the multiplicity of education and training sub- 
systems, institutions and provisions, reflecting a broad 
range of concepts, traditions, values and interests. 
‘Loose’ frameworks introduce a set of comprehensive 
level descriptors to be applied across subsystems, but 
allow sub-frameworks to retain their own principles and 
regulations. ‘Tight’ frameworks differ from this by defin- 
ing uniform specifications for qualifications to be applied 
across sectors. Examples of early versions of frame- 
works in South Africa and New Zealand, which aimed 
to transform national education and training systems, 
illustrate attempts to create tight and ‘one-fit-forall’ solu- 
tions. This created a lot of resistance and led to reas- 
sessment of the role of these frameworks. There is a 
general need for more evidence on how lessons learned 
from the first generation frameworks have been taken 
on board in developing the new generation. It seems (at 
least in Europe) that (simplistic) policy borrowing — heav- 
ily criticised by research — has been replaced by more 
reflective policy learning, paying attention to national 
context and needs. 
‘Loose’ frameworks, emphasising communication 
and transparency, are able to aid and trigger reform. 
This mirrors the ability to mobilise and commit stake- 
holders, rather than to impose ‘one-fit-forall’ rules 
and regulations. 
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(i) 


The (heavily criticised) link between NOFs and neo-lib- 
eral economic policies is hard to detect in current NOF 
developments. In its place is a broader NOF perspective 
which, sometimes in a modest way, addresses a com- 
bination of economic and social equity and sustainable 
development goals. This broader perspective is closely 
linked to the increasingly important role played by life- 
long learning policies, and by the UN in its sustainable 
development goals (2015), articulated as inclusive educa- 
tion and training for all. Offering relevant and high quality 
learning to all, also beyond primary education, is essen- 
tial for sustainable development and reform. 

For an NOF to contribute to these wider policy objec- 
tives it must be embedded in the relevant (national or 
regional) policy context. Standing alone and isolated, 
NOFs are insufficient to support reform and change. For 
frameworks to make a difference, they must interact 
with and add to other policies. A comprehensive NOF 
can be seen as a tool and platform for stakeholder com- 
munication, coordination and cooperation across policy 
areas, levels and institutions. 


(k) The new frameworks aim for overview and support 


(I) 


learning across institutional, sectoral and (sometimes) 
national borders. This brings NOFs close to the objec- 
tives of lifelong and life-wide learning, establishing 
themselves as instruments encouraging and facilitat- 
ing learning careers throughout life and linking formal, 
non-formal and informal learning. National and regional 
qualifications frameworks can support this lifelong learn- 
ing agenda by addressing (through a learning outcomes 
focus) problems related to lack of transparency and frag- 
mentation of provisions and institutions. The potential of 
qualifications frameworks (QF) can only be released the 
moment they start directly to serve individual learners 
and support their lifelong learning pathways. The future 
success of the OFs very much relies on their ability to 
make themselves visible and relevant to end-users. 

Learning outcomes are at the core of national and re- 
gional qualifications frameworks giving stakeholders 
tools for communication, cooperation and coordination 
across Institutions, levels, and sectors, between educa- 
tion and the labour market, and across national borders. 


(m) National case studies show that NOFs are multilevel and 


dynamic tools that evolve over time. They are part of the 
country’s historical, political, institutional and cultural con- 
text and the national qualification and education system 
and labour market. There is a need for more research and 
understanding on how NOFs interact with the national 
qualification systems; what are enablers and implemen- 
tation barriers in particular contexts; and how tensions 
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are resolved. This seems to be especially important 
when discussing NOF development and implementation 
in developed rather than developing countries. Devel- 
oped countries most often have well established nation- 
al qualification systems, strong education institutions, 
trained teachers, established cooperation with social 
partners and other stakeholders; developing countries — 
lacking resources, trained teachers and often with weak 
education institutions — struggle to engage stakeholders, 
reform curricula, and provide capacity building. 

Many NOFs have now completed initial conceptual and 
technical developments. The case studies point to sev- 
eral important factors that shape successful implemen- 
tation. Alongside a solid technical and conceptual foun- 
dation, formal (legal) adoption, institutional structures 
and quality assurance mechanisms, the commitment 
of key stakeholders to the long-term development of 
the framework is of key importance. Evidence from the 
first generation, and also from new frameworks, shows 
that the willingness of powerful stakeholders to use the 
framework is among the most important factors. 


(o) The visibility of the frameworks to end-users, learners 


and workers, is of crucial importance and a condition for 
wider impact. 


(p) The challenges of measuring the impact of qualifica- 


tions frameworks are now at the forefront of discus- 
sions. Experiences so far show that impact assess- 
ment requires agreement on clearly defined baselines 
for assessment. It has to be understood in relation to 
the social, political and institutional contexts in which 
they operate to provide narrative for assessment and 
reflection on why, under what conditions, how and for 
whom the frameworks work. 
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